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Abstract 

The  objective  of  on-line  flagging  in  this  paper  is  to  perform  an  interactive  assessment  of  geosynchronous  satellites 
anomalies  such  as  cross-tagging  of  satellites  in  a  cluster,  solar  panel  offset  change,  etc.  This  assessment  will  utilize  a 
Bayesian  belief  propagation  procedure  and  will  include  an  automated  update  of  the  baseline  signature  data  for  the 
satellite,  while  accounting  for  the  seasonal  changes.  Its  purpose  is  to  enable  an  ongoing,  automated  assessment  of 
satellite  behavior  through  its  life  cycle  using  the  photometry  data  collected  during  the  synoptic  search  performed  by 
a  ground  or  space-based  sensor  as  a  part  of  its  metrics  mission.  The  change  in  the  satellite  features  will  be  reported 
along  with  the  probabilities  of  type  I  and  type  II  errors. 

The  objective  of  adaptive  sequential  hypothesis  testing  in  this  paper  is  to  define  future  sensor  tasking  for  the  purpose 
of  characterization  of  fine  features  of  the  satellite.  The  tasking  will  be  designed  in  order  to  maximize  new 
information  with  the  least  number  of  photometry  data  points  to  be  collected  during  the  synoptic  search  by  a  ground 
or  space-based  sensor.  Its  calculation  is  based  on  the  utilization  of  information  entropy  techniques.  The  tasking  is 
defined  by  considering  a  sequence  of  hypotheses  in  regard  to  the  fine  features  of  the  satellite.  The  optimal 
observation  conditions  are  then  ordered  in  order  to  maximize  new  information  about  a  chosen  fine  feature. 

The  combined  objective  of  on-line  flagging  and  adaptive  sequential  hypothesis  testing  is  to  progressively  discover 
new  information  about  the  features  of  geosynchronous  satellites  by  leveraging  the  regular  but  sparse  cadence  of  data 
collection  during  the  synoptic  search  performed  by  a  ground  or  space -based  sensor. 
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1.0  Introduction 


This  work  is  a  continuation  of  [1].  The  notation  in  this  paper  is  derived  from  [1].  It  builds  upon  the  various  concepts, 
methods,  assumptions  and  procedures  defined  in  [1].  These  include  the  time  slider,  the  two  facet  model,  Brightness 
Ratio,  Cluster-based  evidence,  Model-based  evidence,  Bayesian  belief  propagation,  near-real  time  assessment  of 
change,  Inversion  Model,  Predictive  Model,  and  Statistics  Model.  This  paper  may  be  viewed  as  new  work  on  the 
decision  to  move  the  time  slider  [1],  which  is  required  in  order  to  update  the  baseline  signature  (brightness)  data  for 
a  satellite. 

Reference  1  describes  a  Bayesian  belief  propagation  procedure  for  the  detection  and  resolution  of  cross -tag  between 
two  satellites.  Such  resolution  is  necessary  in  order  to  correctly  associate  new  data  for  each  satellite.  This  paper 
extends  this  data  association  procedure  to  multi-satellite  cross-tag  detection  and  resolution,  including  the 
probabilities  of  false  positive  and  false  negative. 

Fig.  2  -  Fig.  4  show  a  notional  baseline  and  new  data  for  three  satellites  that  comprise  a  notional  cluster.  A  cluster  is 
a  group  of  satellites  that  are  normally  simultaneously  captured  in  a  sensor’s  field  of  view  as  shown  in  Fig.  1.  The 
three  satellites  are  denoted  as  Si,  S2  and  S3,  respectively.  The  baseline  data  is  denoted  with  blue  markers  and  the  new 
data  is  denoted  with  red  markers.  The  data  has  large  gaps,  which  correspond  to  the  daytime  gap  in  data  collection. 
The  same  data  is  plotted  in  Fig.  5  with  respect  to  the  orbit  angle,  which  is  the  same  as  the  longitudinal  phase  angle 
except  that  it  is  computed  with  respect  to  the  orbital  plane  of  the  satellite  instead  of  the  equatorial  plane.  Note  how 
the  brightness  data  for  the  three  satellites  overlaps  with  each  other  making  it  difficult  to  correctly  associate  the  data 
on  the  basis  of  brightness  values  alone.  If  the  satellites  maneuver  during  daytime,  the  new  data  may  be  cross -tagged. 
The  data  in  Fig.  2  -  Fig.  4  may  present  itself  in  six  different  ways  (e.g.  Si  tagged  correctly  but  S2-S3  are  cross- 
tagged,  or  S3  tagged  correctly  but  S1-S2  are  cross-tagged,  or  Si  is  tagged  as  S2,  S2  is  tagged  as  S3  and  S3  is  tagged  as 
Si,  etc.).  The  specific  manner  in  which  the  data  may  present  itself  is  unknown  a  priori.  Accordingly,  the  methods  in 
this  paper  are  designed  to  resolve  the  cross-tag  irrespective  of  how  the  data  presents  itself. 
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Fig.  1.  Sensor  Field  of  View 
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Fig.  2.  Satellite  Si  Apparent  Magnitude  over  Time 


Fig.  3.  Satellite  S2  Apparent  Magnitude  over  Time 
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Fig.  4.  Satellite  S3  Apparent  Magnitude  over  Time 
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Fig.  5.  Signatures  of  the  Three  Satellites  in  the  Cluster 


Reference  1  computes  belief  in  cross-tag  resolution  using  Bayes  theorem,  cluster-based  evidence  and  model-based 
evidence.  This  work  resolves  cross-tag  using  three  methods  (Z-test  for  dependent  data,  classical  sequential  analysis 
and  Brownian  motion  drift  analysis).  It  analyzes  the  sequence  of  values  of  Brightness  Ratio  (rk)  and  P(NOM  I  rk)  for 
this  purpose.  It  also  makes  recursive  use  of  model-base  evidence. 

2.0  Background 

Change  is  defined  as  the  difference  between  the  observed  behavior  and  the  expected  behavior.  The  expected 
behavior  is  defined  by  the  user  on  the  basis  of  prior  data.  Typical  methods  utilized  in  order  to  determine  the 
expected  behavior  are  a  physics-based  model,  statistical  model,  or  data  interpolation.  In  this  work,  we  use  a  physics - 
based  model,  which  is  based  on  the  inversion  of  prior  data.  The  change  is  expressed  in  terms  of  a  ratio,  rk,  where  k 
is  the  observation  number  [1],  which  is  denoted  as  the  Brightness  Ratio.  Specifically: 
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rk  = 


I Mk 


Where  Iok  is  the  observed  single  point  brightness,  IMk  is  the  expected  single  point  brightness.  Under  ideal,  nominal 
conditions,  the  ratio  of  I ok  and  IMk  would  equal  one.  The  values  of  k  <  0  comprise  the  baseline  data  and  k  >  0  is 
new  data  (i.e.  the  green  and  yellow  markers  in  Fig.  6,  respectively).  The  time  slider  is  located  after  k  =  —1,  or  at  the 
end  of  the  baseline  data.  Alternately,  the  time  slider  separates  the  baseline  data  and  new  data.  The  definition  of  rk 
includes  a  subtraction  by  one  so  that  its  expected  value,  E(rk)  «  0  under  ideal  conditions.  The  methods, 
assumptions  and  procedures  are  defined  by  considering  that  E(rk)  =£  0  when  change  occurs.  The  nonzero  value  of 
E(rk )  is  treated  as  a  measure  of  bias  between  the  expected  brightness  and  observed  brightness. 

k  =  - 1 


i 


i 

i 

Fig.  6.  Schematic  of  Time  Slider  Movement 


The  two-facet  model  is  used  as  the  Inversion  Model.  It  represents  a  three-axis  stabilized  satellite  as  two  facets, 
namely  a  body  facet  that  points  to  nadir  and  a  solar  panel  facet  that  tracks  the  sun  as  seen  in  Fig.  7.  This  Inversion 
Model  may  be  considered  a  0th  order  model  that  captures  basic  truth  about  the  satellite,  while  not  explicitly 
accounting  for  the  fine  features  of  the  satellite  (e.g.  antenna,  dishes,  self-occlusions).  The  benefit  of  this  model  is 
that  it  can  be  utilized  to  estimate  the  body  and  panel  optical  behavior  for  any  three-axis  stabilized  satellite.  Its 
shortcoming  is  that  its  accuracy  depends  on  the  complexity  of  the  satellite’s  geometry.  Thus,  there  typically  is  bias 
in  the  values  of  the  Brightness  Ratio,  depending  on  the  observation  conditions  as  follows  in  [1],  and  the  analysis  is 
limited  to  a  subset  of  permissible  orbit  angle  values.  Specifically: 

•  At  small  phase  angles,  the  solar  panel  specular  behavior  dominates  and  the  bias  is  larger. 

•  At  medium  phase  angles  (<  75°),  the  body  diffuse  behavior  dominates.  The  bias  is  small  except  where  there  are 
brightness  contributions  from  features  that  arise  from  self-occlusion  and  body  attachments. 

•  At  phase  angles  close  to  90°,  the  specular  glints  from  body  attachments  dominate  the  character  of  brightness  and 
the  bias  can  be  larger. 

•  The  analysis  is  not  performed  for  phase  angles  greater  than  90°. 
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Fig.  7.  Two-facet  model 


Thus,  the  value  of  E(rk )  is  nearly  zero  for  each  satellite  for  most  orbit  angles.  There  are  regimes  of  orbit  angles 
where  the  Brightness  Ratio  is  large.  These  represent  the  observation  conditions  where  the  Inversion  Model  needs 
improvement,  which  could  be  attained  by  adding  higher  order  terms  to  the  basis  functions  for  the  two  facets  in  order 
to  represent  the  fine  features  or  by  defining  additional  facets  to  represent  the  satellite  geometry.  Note  that  if  there 
was  no  change  in  a  satellite,  sensor  orbit,  and  solar  declination,  the  character  of  E(rk )  is  expected  to  remain 
unchanged  because  the  Inversion  Model  is  ‘invariant’.  If  there  was  change  in  the  satellite,  observation  conditions,  or 
illumination  condition,  the  value  of  E(rk )  can  change.  It  is  necessary  to  assess  if  this  change  is  nominal  (NOM) 
(unchanged  from  the  baseline)  or  anomalous  (ANOM)  (changed  from  the  baseline).  The  value  of  E(rk )  can  become 
ANOM  due  to  cross-tag.  This  is  because,  in  the  calculation  of  the  brightness  ratio,  the  denominator  (or  the  predicted 
value  of  satellite  brightness  by  the  Predictive  Model)  is  for  a  different  satellite. 

The  goal  is  to  detect  change,  characterize  it  as  NOM  or  ANOM  and  provide  its  resolution  as  quickly  as  possible  and 
to  be  able  to  provide  a  measure  of  confidence  for  the  assessment.  In  order  to  detect  change,  we  will  use  three 
different  methods  to  analyze  the  data  sets.  However,  in  each  method,  we  will  be  using  hypothesis  testing  to  evaluate 
our  results.  Each  hypothesis  test  must  have  a  specified  null  hypothesis,  alternative  hypothesis,  test  statistic, 
significance  level  (alpha,  or  probability  of  type  I  error),  and  when  applicable  beta  (probability  of  type  II  error) 
before  beginning.  The  null  hypothesis,  denoted  as  H0 ,  defines  a  state  where  no  change  has  occurred.  The  alternative 
hypothesis,  denoted  as  Ha ,  defines  a  state  where  change  has  occurred.  The  test  statistic  used  in  each  of  the  methods 
involves  the  sample  mean. 

To  assign  a  confidence  level  to  our  calculations,  we  must  designate  specific  values  for  alpha  and  beta.  The  value  of 
alpha  is  equivalent  to  the  probability  of  a  type  I  error.  A  type  I  error  is  when  the  null  hypothesis  is  rejected  and  the 
alternative  hypothesis  is  accepted,  when  in  reality  the  null  hypothesis  is  true.  In  other  words,  the  test  leads  the  user 
to  assess  that  the  satellite  has  changed,  when  the  satellite  state  is  still  NOM  (i.e.  unchanged  from  the  baseline).  A 
false  positive  is  the  event  that  the  test  concludes  that  a  change  has  occurred,  when  in  reality  no  change  has  occurred 
and  the  satellite  state  is  NOM.  Or,  a  false  positive  is  a  type  I  error. 

a  =  level  of  significance  of  the  test  =  P(Type  I  Error ) 

=  P (rejecting  the  null  hypothesis  and  accepting  the  alternative  hypothesis, 
when  in  reality  the  null  hypothesis  is  actually  true) 

Beta  is  defined  to  be  the  type  II  error.  A  type  II  error  is  to  accept  the  null  hypothesis  and  reject  the  alternate 
hypothesis,  when  in  reality  the  alternate  hypothesis  is  true.  In  other  words,  this  is  when  the  hypothesis  test  concludes 
the  satellite  state  to  be  NOM  (i.e.  same  as  the  baseline),  when  in  reality  a  change  has  occurred.  A  false  negative  is 
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when  the  test  concludes  the  satellite  state  to  be  NOM  or  that  nothing  has  changed  as  compared  to  the  baseline,  when 
in  reality  the  satellite  state  has  changed.  Thus,  a  false  negative  is  a  type  II  error. 

/ 3  =  P(Type  II  Error )  =  P (accepting  the  null  hypothesis,  when  in  reality  the  null  hypothesis  is  false) 

3.0  Composite  Hypothesis  Testing 

In  order  to  detect  anomalies,  we  can  utilize  multiple  methods  and  then  combine  their  results  using  a  composite 
hypothesis  test  to  obtain  one  final  resolution.  Three  different  methods  will  be  described  in  Section  4.0  and  any 
combination  of  these  methods  can  be  combined  into  a  composite  hypothesis  test.  In  the  examples  in  Section  8.0,  the 
three  methods  used  in  the  composite  hypothesis  test  will  be  the  Z-test  for  dependent  data  using  the  rk  values,  the  Z- 
test  for  dependent  data  using  the  P(NOMIrfe)  values,  and  the  sequential  analysis  using  either  the  rk  or  A rk  values, 
where  A rk—  rk-  rk_v  As  described  in  the  next  section,  the  Z-test  for  dependent  data  using  the  rk  values  makes  no 
assumptions  for  the  data  and  provides  a  value  for  P(False  Positive).  This  method  is  derived  from  the  physics  based 
model.  The  Z-test  for  dependent  data  using  the  P(NOMIrfc)  values  also  makes  no  assumptions  for  the  data  and 
provides  a  value  for  the  P(False  Positive).  However,  it  is  derived  from  the  Bayesian  belief  propagation.  The 
sequential  analysis  using  either  the  rk  or  A rk  values  assumes  independence  for  successive  points  of  data  and 
provides  a  value  for  the  P(False  Positive)  and  P(False  Negative).  This  method  is  also  derived  from  the  physics  based 
model. 

Once  we  obtain  the  results  for  all  three  tests  for  an  observation,  we  compare  the  results  from  the  different  tests.  If  the 
sequential  analysis  method  determines  we  need  to  continue  sampling,  then  no  decision  is  made,  and  we  wait  for  the 
next  observation.  If  the  sequential  analysis  comes  to  a  conclusion  of  NOM  or  ANOM  and  all  three  tests  agree  a 
satellite  is  NOM  or  ANOM,  then  we  can  make  that  conclusion  as  shown  in  Fig.  8.  However,  if  the  tests  do  not  agree, 
then  we  can  make  no  conclusion  about  that  satellite.  If  all  satellites  in  a  cluster  are  determined  to  be  NOM,  we  can 
update  the  time  slider  as  explained  in  Section  8.9  and  continue  evaluating  observations.  If  a  satellite  is  concluded  to 
be  ANOM,  then  we  will  try  to  resolve  what  changed  in  that  satellite.  One  way  to  do  this  is  by  using  the  cross-tag 
resolution  method  described  in  Section  8.8.  Once  all  cross-tags  and  changes  are  resolved,  we  can  continue  collecting 
data. 


Fig.  8.  Composite  Hypothesis  Testing  Flow  Chart 


Although  Fig.  8  only  illustrates  three  possible  cases,  each  method  can  result  in  multiple  outcomes.  If  we  considered 
all  possible  outcomes  for  each  of  the  three  methods,  the  total  number  of  outcomes  would  be  quite  large.  For  the 
purpose  of  this  assessment,  we  are  only  concerned  with  the  result  when  all  three  tests  produce  the  same  outcome. 
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Thus,  Fig.  8  depicts  the  outcomes  this  paper  will  focus  on.  We  must  also  note  that  there  are  two  separate  methods 
that  can  be  used  for  the  sequential  analysis.  If  rk  is  independent  and  has  an  approximately  normal  distribution  then 
Brownian  motion  drift  analysis  is  used.  If  rk  is  independent  but  does  not  have  an  approximately  normal  distribution 
then  the  classical  sequential  analysis  method  is  used.  This  process  is  illustrated  in  Fig.  9. 


Fig.  9.  Sequential  Analysis  Flow  Chart 
4.0  Hypothesis  Testing  Methods 

The  detection  of  change  is  performed  using  either  the  value  of  rk  or  A rk  and  by  using  the  results  of  the  Bayes  belief 
calculation  (i.e.  P(NOMIrfc)).  There  are  three  methods: 

1 .  Z-test  for  dependent  data  [  1  ] 

2.  Classical  sequential  analysis:  This  is  used  when  rk  is  independent  of  rk_1. 

This  provides  the  ability  to  stop  after  enough  observations  have  been  taken  in  order  to  make  a  decision 
about  the  hypotheses  based  on  the  user-defined  values  of  the  type  I  and  type  II  errors  [2]. 

3.  Brownian  motion  drift  analysis:  This  is  an  application  of  sequential  analysis  that  is  applied  to  the  observed 
process  (rk)  interpreted  as  a  Brownian  motion  in  order  to  determine  whether  or  not  the  drift  is  no  longer 
zero.  The  procedure  is  used  when  A rk  is  independent  of  A rk_t  and  both  have  a  normal  distribution.  This 
analysis  also  provides  the  ability  to  stop  after  a  finite  number  of  observations  based  on  the  user-defined 
values  of  the  type  I  and  type  II  errors  [3]. 

If  there  is  no  change,  the  time  slider  is  moved  forward  and  the  prior  probability  density  function  (pdf)  is  updated. 
This  is  to  enable  an  ongoing  assessment  of  the  satellite  state. 

4.1  Z-test  for  Dependent  Data 

The  Z-test  for  dependent  data  is  a  Central  Limit  Theorem  type  of  statistical  method  that  can  be  used  when  the 
observations  are  not  independent.  Thus,  this  is  a  general  method  that  can  be  used  for  any  set  of  observations. 
The  test  statistic  can  be  approximated  by  a  normal  distribution  under  the  null  hypothesis.  Each  test  statistic  is 
compared  to  the  baseline  by  defining  a  z-score  measure.  The  z-score  represents  the  distance  between  the  sample 
mean  after  pass  k  and  the  baseline  mean  in  terms  of  the  number  of  standard  deviations  away  from  the  sample 
mean.  The  sample  mean,  after  pass  k,  may  be  either  less  or  more  than  the  baseline  mean  in  order  for  the 
absolute  value  of  the  z-score  to  remain  below  a  user-defined  threshold  [1]. 

4.2  Classical  Sequential  Analysis 

Classical  sequential  analysis  is  a  statistical  method  that  evaluates  data  as  it  is  obtained,  as  it  re-computes  the  test 
statistics  each  time  a  new  data  point  is  obtained.  By  using  this  method  to  assess  the  input  data,  the  test  does  not 
require  a  predetermined  sample  size  before  beginning.  This  characteristic  is  unique  to  sequential  analysis  and 
allows  the  user  to  execute  a  decision  earlier  and  with  fewer  observations  as  compared  to  the  Z-test  for 
dependent  data.  The  process  consists  of  computing  a  likelihood  function  and  defining  a  set  of  bounds  based  on 
a  and  /?  to  use  in  evaluating  the  likelihood  output  [2] . 


Classical 

Sequential 

Analysis 


Brownian  Motion 
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4.3  Brownian  Motion  Drift  Analysis 


Brownian  motion  is  a  continuous  limit  of  a  random  walk  process.  The  sequential  analysis  method  can  be 
applied  to  Brownian  motion  to  test  whether  or  not  the  drift  of  the  Brownian  motion  process  is  zero.  This  method 
is  useful  because  it  evaluates  data  as  it  is  collected  and  allows  the  user  to  make  a  decision  once  a  fixed  bound 
has  been  reached.  The  Brownian  motion  approach,  however,  has  a  specific  set  of  requirements  that  must  be 
verified  before  being  used  for  testing  [3]. 

5.0  Hypothesis  Testing  Assumptions 

5.1  Z-test  for  Dependent  Data 

In  order  to  receive  accurate  results  using  the  Z-test  for  dependent  data,  there  are  certain  conditions  that  must  be  true 
about  the  data  set  that  will  be  evaluated: 

•  Large  sample  size  n  for  the  baseline  data. 

•  The  data  points  are  assumed  to  be  sampled  from  the  same  population  distribution. 

5.2  Classical  Sequential  Analysis 

In  order  to  receive  accurate  results  using  classical  sequential  analysis,  there  are  certain  conditions  that  must  be  true 
about  the  data  set  that  will  be  evaluated: 

•  rk  is  independent  of  rk_x  in  the  baseline  data. 

•  This  method  does  not  require  the  data  set  to  have  a  standard  normal  distribution. 

5.3  Brownian  Motion  Drift  Analysis 

In  order  to  apply  methods  applicable  to  Brownian  motion  to  rk ,  there  are  certain  conditions  that  must  be  true  about 
the  data  set  that  will  be  evaluated: 

•  A rk  is  independent  of  Ar/^  in  the  baseline  data. 

•  Standard  Normal  density  (pdf)  of  A r  values 

•  rk(0)  =  0 

•  rk(t),  0  <  t  <  oo  is  a  continuous  function  of  t  (or  time).  Note  that  each  observation  k  is  associated  with  its 
monotonically  increasing  value  of  time  at  which  the  observation  was  taken. 

6.0  Hypothesis  Testing  Procedures 
6.1  Z-test  for  Dependent  Data 

While  describing  the  Z-test  for  dependent  data,  the  following  description  considers  the  rk  values,  although  the 
procedure  is  the  same  when  using  the  value  of  P(NOMI  rk). 

•  In  order  to  perform  the  Z-test  for  dependent  data,  the  null  hypothesis  and  alternate  hypothesis  are  defined  as 
follows.  The  null  hypothesis  defines  a  state  when  no  change  has  occurred.  The  alternative  hypothesis  defines  a 
state  where  change  has  occurred  in  the  mean  of  rk  values.  The  estimated  value  of  the  population  mean  E(rk)  is 
denoted  as  fik,  which  is  the  sample  mean.  The  hypothesis  test  is  defined  in  order  to  assess  if  /ik> 0  differs  from 
the  baseline  mean,  fik<0: 


Null  hypothesis  =  H0  :  fik>0  =  fik< 0 
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Alternative  hypothesis  =  Ha  :  fik> 0  ^  M/c<o 


The  Z-test  for  dependent  data  does  require  us  to  pick  a  significance  level  a.  We  will  use  a  =  .05.  However,  the 
Z-test  for  dependent  data  does  not  allow  us  to  specify  p.  Therefore,  we  will  not  know  our  type  II  error  for  these 
calculations  and  cannot  accept  our  null  hypothesis  at  any  point  for  this  method  with  a  handle  of  the  probability 
of  doing  so  in  error. 

The  Z-test  for  dependent  data  algorithm  starts  with  the  given  baseline  data  containing  n  observations,  shown  as 
the  green  segment  in  Fig.  6.  These  data  points  are  considered  to  be  before  the  user-defined  reference  time,  to. 
The  mean  is  calculated  for  the  values  of  this  data  set,  which  provide  the  baseline  against  which  the  new  data  is 
compared: 


Baseline  mean  =  /J-k<0,  which  is  estimated  by  the  sample  mean  ~^Yik=-nrk 

Since  the  successive  values  of  rk  can  be  dependent,  the  population  variance  o2  is  adjusted  using  what  we  will 
call  the  “dependent  variance”.  This  dependent  variance  reflects  the  alpha-mixing  parameter,  a,  which  specifies 
how  many  previous  passes  on  which  each  value  has  significant  dependence.  The  dependent  variance  is 
comprised  of  the  sample  variance  and  a  term  that  involves  covariance,  which  is  calculated  using  the  alpha 
mixing  procedure  [4] .  Specifically: 

Dependent  variance  =  population  variance  +  2*  (covariance-related  term) 

The  covariance-related  term  is  estimated  by: 

( °k< of  =  Sk< o  +  2  *  ^Ifc=_„l2“=i[(nk  -  /Jk<o)Oik  -  /4< o)]L 

Sk <0  =  Var(Xk< 0)  =  -^-^k=-n(Tk  ~  M/c< o)  2  >  which  estimates  the  variance  of  the  baseline  rk  values 

The  alpha  mixing  calculations  commence  when  a  sufficient  amount  of  new  data  is  received.  The  number  of 
passes  that  constitute  sufficient  new  data  may  be  a  fraction  of  the  data  size  in  the  baseline  (e.g.  when  new  data 
is  collected  that  spans  a  full  night).  When  new  data  is  received  for  passes  0  to  k  (shown  in  yellow  color  in  Fig. 
6),  the  data  that  spans  from  pass  (k-n)  to  pass  (k)  is  utilized  to  perform  the  assessment.  The  data  from  passes  (k- 
n)  to  0  is  from  the  baseline.  This  corresponds  to  the  use  of  a  sliding  window  of  the  last  n  observations  for  the 
alpha  mixing  analysis. 


For  each  new  pass  k,  the  value  of  rk  is  computed  first  and  then  used  for  the  Z-test  for  dependent  data 
calculation.  The  mean  is  estimated  for  the  rk  for  the  set  of  values  from  passes  (n-k)  to  k: 

Current  sample  mean  after  pass  k  =  estimate  of  iik>0  =  fk  =  ^Ta=k-n+i  rk 


The  estimate  of  the  current  mean  after  pass  k,  rk,  is  compared  with  the  baseline  by  defining  a  z-score  measure 
(or  standard  score  measure),  which  is  denoted  as  r]k  [4]: 


Z-score  r]k  = 


rk~  Mk< o 
°k<o. 


The  z-score  represents  the  distance  between  the  sample  mean  after  pass  k  and  the  baseline  mean  in  terms  of  the 
number  of  standard  deviations.  The  sample  mean  after  pass  k  may  be  less  or  more  than  the  baseline  mean  in 
order  for  the  absolute  value  of  the  z-score  to  remain  below  a  user-defined  threshold. 

Each  new  observation  for  which  we  do  not  reject  the  null  hypothesis,  we  continue  to  assume  that  the  satellite  is 
NOM  (i.e.,  unchanged  from  the  baseline)  and  continue  to  evaluate  new  observations.  When  there  is  an 
observation  that  does  not  satisfy  the  null  hypothesis,  it  implies  that  the  mean  has  changed  in  a  statistically 


AMOS  Technical  Conference  2015 


10 


significant  amount  as  compared  to  the  baseline,  which  prompts  us  to  reject  the  null  hypothesis  and  accept  the 
alternative.  Therefore,  we  can  conclude  the  satellite  is  different  from  its  state  in  the  baseline  data  (i.e.  the 
occurrence  of  change)  and  the  time  slider  cannot  be  forwarded  up  to  the  current  pass  k.  Once  we  have  obtained 
n  new  observations  for  which  we  do  not  reject  the  null  hypothesis,  the  time  slider  can  be  moved  forward  to  the 
current  pass  k. 


Fig.  10.  Z-test  for  Dependent  Data  Flow  Chart 


6.2  Classical  Sequential  Analysis 

The  classical  sequential  analysis  method  will  proceed  by  using  the  7\  values;  however,  other  values  may  also  be  used 
as  long  as  they  are  independent.  The  classical  sequential  analysis  does  not  require  the  data  set  to  have  a  standard 
normal  distribution. 

•  As  described  in  Section  6.1,  this  method  also  requires  the  creation  of  a  set  of  baseline  r  values.  Once  the 
baseline  has  been  defined,  the  baseline  mean  and  standard  deviation  must  be  computed  for  future 
calculations. 

•  The  classical  sequential  analysis  method  does  require  a  predetermined  a  and  p.  The  test  also  requires  a 
predefined  8  that  is  fixed  to  be  greater  than  zero,  where  delta  is  the  number  of  standard  deviations  the 
sample  mean  is  away  from  the  null  hypothesis  mean  0.  For  our  examples,  a  =  0.05,  p  =  0.05,  and  8  =  1.0. 
Because  this  test  allows  us  to  determine  a  and  p,  we  can  quantify  the  P(type  I  error),  P(type  II  error)  and  be 
able  to  accept  the  null  hypothesis  when  necessary. 

•  To  be  able  to  properly  use  the  classical  sequential  analysis  test,  a  null  hypothesis  and  alternative  hypothesis 
must  also  be  defined.  The  null  hypothesis  describes  a  state  where  no  change  has  occurred.  The  alternative 
hypothesis  describes  a  state  where  a  change  has  occurred  that  is  larger  than  the  predefined  8.  Both 
hypotheses  are  defined  below. 

Null  hypothesis  =  H0:  p  =  po 
Alternative  hypothesis  =  Ha:  |^-^|  >  8 

•  Before  the  method  can  be  evaluated,  a  user-defined  bound  must  be  established.  The  lower  bound  is  denoted 
as  B  and  is  defined  by 

B  =-£- 

1  -a 
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The  upper  bound  is  denoted  as  A  and  is  defined  by 


a 

•  Once  a  new  set  of  data  is  received,  the  classical  sequential  analysis  method  uses  a  likelihood  function  to 
evaluate  the  data  set  at  each  observation  k.  The  likelihood  function  has  a  closed  form  expression  if  rk  is 
normally  distributed;  it  has  to  be  numerically  computed  otherwise.  In  this  work,  the  likelihood  function  is 
computed  using  the  following  equation,  which  is  valid  for  normally  distributed  rk: 

U  =  \*  e>2(ef^-^  +  e-fzcr*-^) 

•  After  the  likelihood  function  is  computed,  the  output  is  evaluated  according  to  the  prefixed  bounds.  If  the 
likelihood  function  E  (B,  A)  then  the  process  continues  sampling.  If  the  likelihood  function  is  less  than  or 
equal  to  B,  then  the  null  hypothesis  is  accepted  and  thus,  we  conclude  that  no  change  has  occurred.  If  the 
likelihood  function  is  greater  than  or  equal  to  A,  then  the  alternative  hypothesis  is  accepted,  and  thus,  we 
conclude  that  a  change  has  occurred. 


Fig.  11.  Classical  Sequential  Analysis  Flow  Chart 


6.3  Brownian  Motion  Drift  Analysis 

As  described  early,  Brownian  motion  is  a  specific  sequential  analysis  method,  so  its  process  is  very  similar  to  that  of 
the  classical  sequential  analysis  method.  For  our  examples,  the  main  difference  between  Brownian  motion  and  the 
classical  sequential  analysis  is  that  the  Brownian  motion  baseline  is  executed  using  the  Ark values.  The  creation  of 
the  baseline  is  identical  to  that  described  in  Section  6.1. 

•  Once  the  rk  baseline  has  been  created,  the  Ark values  are  computed  using  the  equation  listed  below. 


Ark  =rk  -  rk_! 


After  the  Ark  baseline  is  generated,  the  mean  and  standard  deviation  of  the  baseline  is  calculated  for  a  later 
use. 
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Brownian  motion  does  require  a,  P  and  8  to  be  set  prior  to  testing.  For  this  method,  delta  is  defined  as  the 
number  of  standard  deviations  that  mean  drift  pis  away  from  null  hypothesis  mean  drift  0.  The  Brownian 
motion  examples  have  a  =  0.05,  p  =  0.05  and  8  =  1.0. 

A  null  hypothesis  and  alternative  hypothesis  must  also  be  defined  in  order  to  proceed  with  Brownian 
motion.  The  null  hypothesis  describes  a  state  where  no  change  has  occurred.  The  alternative  hypothesis 
describes  a  state  where  a  change  has  occurred.  Both  hypotheses  are  defined  below. 

Null  hypothesis  =  H0:  p  =  0 

Alternative  hypothesis  =  Ha:  |p|  >  8  *  a 

Before  the  method  can  be  evaluated,  a  user-defined  bound  must  be  established.  The  lower  bound  is  denoted 
as  A  and  is  defined  by 


The  upper  bound  is  denoted  as  B  and  is  defined  by 


B=^ 

a 

Once  a  new  set  of  data  is  received,  the  Brownian  motion  drift  method  standardizes  each  rk  by  subtracting 
each  rk  by  the  value  at  observation  0  and  then  dividing  by  the  standard  deviation  of  the  baseline.  This 
adjusted  r  value  is  denoted  below. 

rk  ~  r0 

rk=~ - 

° baseline 

This  method  also  uses  a  likelihood  function  to  evaluate  the  data  set  at  each  observation  k.  This  likelihood 
function  is  computed  using  the  following  equation  for  two-sided  drift  analysis  as  explained  in  Section 
6.3.1: 


k82 

L„  =  0.5  *  e~  *  (e(5*f)  + 

After  the  likelihood  function  is  computed,  the  output  is  evaluated  according  to  the  prefixed  bounds.  If  the 
likelihood  function  E  (A,  B)  then  the  process  continues  sampling.  If  the  likelihood  function  is  less  than  or 
equal  to  A,  then  the  null  hypothesis  is  accepted  and  thus,  we  conclude  that  no  change  has  occurred.  If  the 
likelihood  function  is  greater  than  or  equal  to  B,  then  the  alternative  hypothesis  is  accepted,  and  thus,  we 
conclude  that  a  change  has  occurred. 
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Fig.  12.  Brownian  Motion  Drift  Analysis  Flow  Chart 
6.3.1  Two-sided  Drift  Analysis 

Reference  3  presents  a  hypothesis  test  to  detect  change  in  drift  for  Brownian  motion.  If  p0  denotes  the  drift 
corresponding  to  the  null  hypothesis  and  pi  denotes  the  drift  corresponding  to  the  alternative  hypothesis,  then  the 
likelihood  ratio  function 

L(t,W(ty,  /zo, Mi)  = 

is  used  to  test  H0:  p=  p0  vs.  Ha:  p=  pi  for  values  p0  <  pi  [3].  Our  process  rk  is  transformed  to  the  correct  scale  to  be  a 
Brownian  motion,  and  this  likelihood  ratio  L(t,  W (t);  p0>  Mi)  can  be  approximated  discretely  by 

4  (Mo,  Mi)  =  e[(*1-'*o)*r*4(^-^)] 

In  our  case,  p0=  0,  referring  to  the  null  hypothesis  case  of  “no  drift”,  which  we  are  equating  to  the  NOM  state. 
However,  we  are  interested  in  a  test  for  H0:  p  =  0  vs.  Ha:  |p|  >pi  for  some  value  pi>  0. 

Note  that  our  process  r  was  normalized  to  be  a  Brownian  motion  (with  scale  a  =1),  and  we  are  testing  the  alternative 
hypothesis  |  p  |>  8. 

Following  the  idea  presented  in  [2]  to  extend  a  likelihood  ratio  test  for  a  1  -sided  alternative  to  a  test  for  a  2-sided 
alternative,  we  compute  a  new  likelihood  ratio  function  as  follows.  The  numerator  is  the  sum  of  the  likelihood 
functions  (i.e.  joint  densities)  associated  with  p  =  5  and  p  =  -5,  each  multiplied  by  probability  0.5.  The  denominator 
is  the  likelihood  function  associated  with  p  =  0.  Dividing  the  denominator  into  both  numerators  results  in  the 
expression  being  the  sum  of  two  likelihood  ratio  functions  corresponding  to  the  alternative  hypotheses  p=  8  and  p  =- 
5,  respectively. 

That  is,  Lk(fi0,  px)  for  our  2-sided  case  with  po=0  and  pi  =+/-  5  becomes 

4 0*0,  Ml)  =  -5*  4(0,5)+  .5*  4  (0,-5) 

=  .5*  eM)*rk-|.52]+  5*  e[{S>rk-\*8*\ 

=  .5  *  *  (e[-5trk]  +  e[<5’rfcl) 


, which  is  the  formula  mentioned  above. 
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6.4  A  Note  on  the  Use  of  the  Brightness  Ratio  during  Hypothesis  Testing 


The  Brightness  Ratio  is  defined  in  Section  2.0.  Note  rk  is  not  symmetric.  For  example,  if  Iok  =  5  *  lMk  then  rk  = 

4 

4  but  if  IMk  =  5  *  Iok ,  then  rk  =  —  =  —0.8.  The  range  of  values  we  consider  to  be  acceptable  for  rk  is  (-0.6, 
1.5).  Although  the  r  values  are  asymmetric,  this  characteristic  is  only  of  concern  during  the  classical  sequential 
analysis  test.  This  is  the  case  because  the  Z-test  for  dependent  data  has  a  test  statistic  that  is  normal  due  to  the 
Central  Limit  Theorem  for  Dependent  Data,  irrespective  of  distribution.  Similarly,  the  Brownian  motion  drift 
analysis  is  computed  only  on  the  delta  r  values,  which  can  be  independent  and  normally  distributed  regardless  of 
the  probability  distribution  of  the  r  values.  Thus,  the  classical  sequential  analysis  method  applied  to  r  is  the  only 
method  that  may  be  affected  by  the  asymmetry.  However,  classical  sequential  analysis  does  not  require  the  data  set 
to  be  sampled  from  a  normal  distribution.  For  future  work  in  this  case,  the  likelihood  function  used  in  the  classical 
sequential  analysis  will  be  derived  from  the  population  distribution  via  an  empirical  probability  density  function. 

7.0  Cross-tag  Resolution  Method 

7.1  Overview 

Once  the  results  from  the  three  methods  have  been  obtained,  we  can  begin  analysis.  If  all  three  methods  determine  a 
satellite  is  ANOM,  then  we  can  conclude  the  satellite  has  changed  in  some  way.  One  way  the  satellite  can  change  is 
due  to  a  cross-tag  with  one  of  its  peer  satellites.  This  means  the  satellite  has  been  miss  identified  as  another  satellite 
in  its  cluster.  This  method  helps  to  resolve  those  situations. 

7.2  Methods 

Suppose  the  three  methods  have  determined  at  least  one  of  the  satellites  is  ANOM.  Then,  we  must  try  to  determine 
the  change  in  that  satellite.  If  two  satellites  are  determined  to  be  ANOM  by  all  three  tests  or  one  satellite  is 
determined  to  be  ANOM  and  the  other  satellite’s  state  cannot  be  determined,  then  we  can  use  the  cross-tag 
resolution  method  to  determine  if  the  cause  of  ANOM  is  due  to  a  cross-tag.  When  two  satellites  are  determined  to  be 
ANOM  by  at  least  one  test,  we  can  take  the  new  data  from  the  two  satellites  and  swap  them.  Thus,  the  baseline  of 
the  first  satellite  is  now  paired  with  the  new  data  of  the  second  satellite,  while  the  baseline  of  the  second  satellite  is 
paired  with  the  new  data  of  the  first  satellite.  The  three  methods  can  be  run  again  on  these  new  pairs  of  data  to  see  if 
this  swap  resolved  the  cross-tag. 

7.3  Assumptions 

The  present  work  assumes  if  a  cross-tag  does  occur  the  cross-tag  occurred  during  the  daytime  gap.  Fig.  2  -  Fig.  4 
illustrate  the  reason  for  this  assumption.  The  baseline  data  for  each  satellite  concludes  at  the  end  of  a  night’s 
observations,  and  the  new  day  begins  at  the  start  of  the  next  night.  Therefore,  when  the  three  methods  are  performed, 
a  result  is  obtained  and  applied  to  all  the  new  data  since  the  beginning  of  that  night.  Thus,  we  conclude  something 
changed  during  the  daytime  gap. 

7.4  Procedures 

After  the  composite  hypothesis  test  has  determined  at  least  one  satellite  to  be  ANOM,  we  must  try  to  determine  the 
changes  in  the  satellite.  If  there  is  a  second  satellite  which  is  determined  to  be  ANOM,  or  no  conclusion  could  be 
reached,  then  we  can  use  the  two  not  NOM  satellites  in  this  cross-tag  resolution  method.  The  procedure’s  steps  are 
listed  in  order  below. 

1.  Swap  the  new  observations  for  the  two  satellites,  Si  and  S2.  Thus,  the  baseline  of  satellite  Si  is  now  paired 
with  the  new  data  of  satellite  S2,  while  the  baseline  of  S2  is  paired  with  the  new  data  of  Si. 

2.  Compute  the  prior  distribution  for  the  values  being  used:  rk ,  A rk,  or  P(NOM  I  rk). 

3.  Perform  the  three  methods  again  as  described  in  Sections  6.1  and  6.2. 
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4.  Analyze  the  new  results.  If  all  three  tests  now  conclude  NOM  for  either  satellite,  say  Si,  then  there  was  a 
cross-tag  present.  If  one  of  the  satellites  is  still  ANOM,  say  S2,  and  there  was  a  third  satellite,  S3,  in  the 
cluster  that  was  not  NOM,  then  we  can  perform  another  swap.  We  can  swap  the  new  data  of  S2  after  the 
first  swap  with  the  new  data  of  S3  by  completing  steps  1-4  again.  However,  if  the  results  still  conclude  not 
NOM  for  both  satellites,  then  we  cannot  determine  if  there  was  a  cross-tag  and  it  is  likely  that  the  cause 
could  be  from  a  different  cross-tag  scenario  or  that  a  different  change  had  occurred  in  the  satellite. 

Once  all  the  possible  cross-tag  situations  have  been  tested  for  all  satellites  in  the  cluster  that  were  determined  to  not 
be  NOM  by  the  composite  hypothesis  test,  we  have  a  better  understanding  of  which  satellites  were  cross -tagged  and 
have  resolved  the  cross-tag  so  we  are  associating  the  data  to  the  right  object. 

7.5  Confidence  Assessment 

To  be  able  to  utilize  the  cross-tag  resolution  method  described  above,  the  composite  hypothesis  test  used  to 
determine  whether  or  not  we  believe  that  a  particular  satellite  has  changed  from  what  is  believed  to  be  its  NOM  state 
must  be  assigned  a  level  of  confidence.  Suppose  that  each  hypothesis  test  is  performed  at  significance  level  a  and 
that  the  null  hypothesis  H0  reflects  that  the  satellite  is  NOM.  Significance  level  a  means  that  there  is  only  a 
probability  that  the  test  will  incorrectly  conclude  that  the  satellite  is  ANOM,  when  in  reality  it  is  NOM;  i.e.  a  =  P 
(test  rejects  H0  I  H0  is  true). 

Recall  that  if  at  least  one  of  the  tests  concludes  that  the  satellite  is  ANOM,  we  will  not  conclude  that  the  satellite  is 
NOM.  It  is  of  interest  to  know  the  probability  of  type  I  error  of  this  sort  of  conclusion.  To  this  end,  we  compute  the 
probability  that  we  will  not  conclude  that  the  satellite  is  NOM,  when  in  reality  the  satellite  is  NOM  as  follows: 

P(at  least  one  of  the  3  tests  incorrectly  conludes  that  the  satellite  is  ANOM  \  satellite  is  NOM ) 

=  P(  null  hypothesis  is  rejected  in  at  least  one  of  the  3  hypothesis  tests  \H0  is  true ) 

=  P  ^  |^J  {test  i  rejects  H0] 

= 

=  3  a 


.  is  trnipA  <  V  PC test  i 


_ _ A _ T  T 


This  means  that  the  composite  test  for  whether  or  not  the  satellite  is  NOM  using  the  group  of  3  tests  performed 
individually  at  level  a  has  an  overall  significance  level  of  less  than  or  equal  to  3a. 

For  the  current  case,  we  perform  the  3  tests  on  a  single  satellite  at  level  0.05.  Moreover,  suppose  that  at  least  1  of 
the  3  tests  results  in  a  rejection  of  the  null  hypothesis  that  the  satellite  is  NOM.  If  we  use  this  criterion  of  “at  least 
one  rejection”  necessary  to  conclude  that  the  satellite  is  not  NOM,  then  our  method  has  reached  the  conclusion:  “we 
cannot  conclude  that  the  satellite  is  NOM”.  Using  the  above  calculation,  we  recognize  that  our  method  has  less  than 
or  equal  to  0.15  probability  of  failing  to  conclude  that  the  satellite  is  not  NOM,  when  in  reality  it  is  NOM. 

Additionally,  we  are  interested  in  knowing  the  probability  that  the  composite  test  method  correctly  concludes  that 
the  satellite  is  NOM,  which  would  be  the  case  if  all  three  tests  conclude  to  not  reject  the  null  hypothesis  of  NOM.  In 
other  words,  what  is  the  P(all  3  tests  correctly  result  in  non-rejection  of  null  hypothesis  that  satellite  is  NOM  I 
satellite  is  NOM)?  To  compute  this  number,  we  use  the  complement  rule  and  the  above  computation  as  follows: 

P(  all  3  tests  correctly  result  in  non  rejection  of  the  null  hypothesis  that  the  satellite  is  NOM  \  satellite  is  NOM ) 

=  1  —  P(null  hypothesis  is  rejected  in  at  least  one  of  the  3  tests  of  H0  \H0  is  true ) 
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=  1  —  P  ^  |^J  {test  i  rejects  H0] 


H0  is  true )  >  y  Pjtest  i  rejects  H0  \  H0  is  true ) 


=  1-3  a 

In  other  words,  we  perform  the  3  tests  on  a  single  satellite  at  level  0.05.  Moreover,  suppose  that  none  of  the  3  tests 
results  in  a  rejection  of  the  null  hypothesis  that  the  satellite  is  NOM.  Using  our  composite  testing  methods,  we  are 
led  to  conclude  that  the  satellite  is  NOM.  By  the  above  computation,  we  recognize  that  our  method  has  at  least  a 
0.85  probability  of  correctly  concluding  that  satellite  is  NOM. 

8.0  Examples 

To  illustrate  the  three  methods  described  in  Section  4.0,  we  will  consider  the  three  satellite  cluster  shown  previously 
in  Fig.  5.  The  three  satellites  denoted  as  Si,  S2,  and  S3,  and  their  signatures  are  displayed  in  Fig.  13  -  Fig.  15.  This  is 
the  same  data  as  in  Fig.  2  -  Fig.  4.  The  observations  were  simulated  as  taken  from  a  space-based  sensor.  The 
observations  for  Si  span  7  days,  while  the  observations  for  S2  and  S3  only  span  5  days.  The  number  of  days  of 
observation  can  be  visualized  by  counting  the  daytime  gaps  in  Fig.  2  -  Fig.  4. 

Each  data  set  was  divided  into  two  parts.  The  first  portion  was  used  to  simulate  a  set  of  baseline  data.  A  baseline  is 
data  which  has  been  reviewed  by  a  human  or  passed  a  collection  of  statistical  tests  in  order  to  verify  which  satellite 
the  observations  belong  to  and  that  the  satellite’s  state  is  nominal  (i.e.,  NOM).  The  second  portion  of  data  represents 
the  new  incoming  observations  for  that  satellite.  The  baseline  is  used  to  determine  what  results  we  can  expect  for  a 
particular  satellite,  and  these  expectations  can  be  used  to  verify  the  current  state  of  the  satellite  from  the  new 
observations.  Fig.  13  -  Fig.  15  display  the  three  data  sets  used  for  the  examples.  The  blue  points  in  each  graph 
represent  the  observations  used  as  the  baseline  for  that  satellite,  while  the  red  points  represent  the  new  observations 
which  are  being  tested  with  each  of  the  three  methods. 
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Fig.  13.  Satellite  Si  Signature  Plot 
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S2  Signature  Plot 
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Fig.  14.  Satellite  S2  Signature  Plot 


Fig.  15.  Satellite  S3  Signature  Plot 


Once  the  baseline  and  the  new  data  sets  were  created  for  each  satellite,  the  baseline  values  were  further  reduced  by 
randomly  selecting  points  from  the  data.  Since  there  is  no  guarantee  how  much  data  will  be  provided,  we  want  to 
use  these  tests  on  the  minimum  amount  of  data,  so  we  generate  sparse  baseline  data  sets  of  31  or  32  points.  This 
way,  even  when  using  the  differences  in  the  rk  values,  we  will  still  have  30  points  and  the  Central  Limit  Theorem  for 
Dependent  Data  will  be  valid  for  the  Z-test  method.  To  generate  the  sparse  data,  we  sorted  by  orbit  angle.  We  took 
the  total  number  of  points  divided  by  30  to  determine  how  often  to  select  an  observation.  Then,  we  randomly 
selected  a  point  that  frequently  to  get  31  or  32  observations.  The  resulting  signature  graphs  for  the  three  satellites  are 
shown  in  Fig.  16  -  Fig.  18.  This  reduced  data  was  utilized  in  order  to  perform  the  change  detection  and  data 
association. 
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St  Sparse  Signature  Plot 
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Fig.  16.  Satellite  Si  Sparse  Signature  Plot 


S2  Sparse  Signature  Plot 
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Fig.  17.  Satellite  S2  Sparse  Signature  Plot 
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S3  Sparse  Signature  Plot 


Orbit  Angle  (degrees) 

Fig.  18.  Satellite  S3  Sparse  Signature  Plot 


From  the  three  data  sets,  we  can  construct  6  more  data  sets.  From  these  9  data  sets,  we  can  create  6  different 
potential  cross-tag  situations  which  are  displayed  in  Table  1.  The  analysis  of  these  six  cases  and  their  resolutions  are 
presented  in  section  8.0.  Each  example  describes  how  a  cross-tag  anomaly  is  resolved  to  allow  for  correct  data 
association.  Section  9.0  describes  an  efficient  way  to  collect  data  using  maximum  information  entropy. 


Table  1.  Possible  Cross-tag  Situations 


Example 

Satellite  Si: 

Satellite  S2: 

Satellite  S3: 

1 

NOM 

NOM 

NOM 

2 

Cross-tagged  as  S2 

Cross-tagged  as  Si 

NOM 

3 

Cross-tagged  as  S3 

NOM 

Cross-tagged  as  Si 

4 

NOM 

Cross-tagged  as  S3 

Cross-tagged  as  S2 

5 

Cross-tagged  as  S2 

Cross-tagged  as  S3 

Cross-tagged  as  Si 

6 

Cross-tagged  as  S3 

Cross-tagged  as  Si 

Cross-tagged  as  S2 

8.1  Method  Parameters 

8.1.1  Z-test  for  Dependent  Data 

The  Z-test  used  here  is  based  on  a  Central  Limit  Theorem  for  dependent  data.  There  are  no  population  assumptions 
necessary,  but  if  the  sample  size  is  sufficiently  large,  the  sample  mean  values  have  an  approximately  normal 
distribution  from  the  Central  Limit  Theorem.  Therefore,  as  long  as  we  have  at  least  30  values,  it  is  reasonable  to 
apply  the  Central  Limit  Theorem  for  dependent  data.  For  the  examples,  we  will  use  the  Z-test  for  dependent  data  on 
both  the  r  values  and  the  P(NOMIrk)  values. 

8.1.2  Classical  Sequential  Analysis 

The  classical  sequential  analysis  method  has  one  main  assumption,  and  it  is  that  the  observed  sequence  values  are 
independent.  To  determine  if  the  rk  values  or  the  Ark  values  are  independent  or  not,  we  use  the  sample  correlation. 
We  also  create  a  histogram  of  the  values  to  evaluate  if  it  is  approximately  normal.  If  the  random  variables  appear  to 
be  uncorrelated  with  an  approximately  normal  distribution,  then  we  will  assume  independence  and  can  use  those 
values  for  the  classical  sequential  analysis  method.  The  sparse  baseline  r  values  are  shown  in  Fig.  19  -  Fig.  21. 
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Fig.  19.  Satellite  Si  Sparse  r  Baseline 
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Fig.  20.  Satellite  S2  Sparse  r  Baseline 
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Fig.  21.  Satellite  S3  Sparse  r  Baseline 


To  determine  if  the  rk  and  rk+i  values  are  correlated,  we  first  calculated  the  correlation  coefficient  for  each  satellite 
for  the  sparse  baseline  values.  Then,  we  used  a  t-test  with  a  =  0.05  on  the  null  hypothesis  that  the  correlation 
coefficient  is  equal  to  zero.  The  results  for  the  r  values  are  in  Table  2,  while  the  results  for  the  Ar  values  are  in  Table 
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3.  If  the  correlation  coefficient  is  within  the  non-rejection  region  of  the  t-test  determined  by  a  =  0.05,  then  we  cannot 
reject  the  null  hypothesis  that  the  correlation  coefficient  is  zero  and  therefore,  that  the  values  are  uncorrelated. 

Table  2.  r  Value  Correlation  Results 


Satellite 

Correlation 

Coefficient 

Test  statistic 
value 

Non-rejection 

Region 

Conclusion 

Si 

-0.3521 

-2.0259 

(-2.045,  2.045) 

uncorrelated 

s2 

0.0007 

0.0035 

(-2.048,  2.048) 

uncorrelated 

S3 

0.3005 

1.6969 

(-2.045,  2.045) 

uncorrelated 

Table  3.  Ar  Value  Correlation  Results 


Satellite 

Correlation 

Coefficient 

Test  statistic 
value 

Non-rejection 

Region 

Conclusion 

Si 

-0.7021 

-5.2176 

(-2.048,  2.048) 

correlated 

s2 

-0.6946 

-5.0169 

(-2.052,  2.052) 

correlated 

S3 

-0.1704 

-0.9313 

(-2.048,  2.048) 

uncorrelated 

Looking  at  Table  2,  we  see  that  by  the  t-test  the  baseline  r  values  for  all  three  satellites  are  uncorrelated.  However, 
Table  3  shows  us  that  the  Ar  values  are  only  uncorrelated  for  S3.  Thus,  we  can  use  the  classical  sequential  analysis 
on  all  three  satellites  for  their  r  values,  but  we  can  only  use  it  on  the  Ar  values  for  S3. 

The  data  being  uncorrelated  is  a  necessary  condition  for  the  data  to  be  independent,  but  to  assure  the  data  is 
independent,  we  must  also  satisfy  a  sufficient  condition,  such  as  the  data  is  normally  distributed.  To  determine  if  the 
sparse  baseline  r  values  are  approximately  normal,  we  plotted  the  histogram  of  the  r  values  from  Fig.  19  -  Fig.  21. 
Fig.  22  shows  the  histogram  of  the  baseline  values  for  Si,  and  the  histogram  supports  normality.  Fig.  23  displays  the 
r  value  histogram  for  S2,  and  it  also  supports  normality.  Fig.  24  illustrates  the  r  value  histogram  for  S3.  It  does  not 
appear  normal.  We  will  proceed  with  the  assumptions  that  Si  and  S2  are  normal  distributed. 
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Fig.  22.  Histogram  of  Si  Baseline  r  Values 
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Fig.  23.  Histogram  of  S2  Baseline  r  Values 
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Fig.  24.  Histogram  of  S3  Baseline  r  Values 


The  full  baseline  for  the  Ar  values  is  displayed  in  Fig.  25,  and  their  histogram  is  shown  in  Fig.  26  to  better 
demonstrate  that  the  Ar  values  for  S3  are  approximately  normal.  Therefore,  we  can  assume  the  Ar  values  for  S3  are 
independent  and  use  them  for  the  classical  sequential  analysis  method. 
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S3  F ull  Baseline  Ar  Values 


Fig.  25.  Satellite  S3  Full  Ar  Baseline 
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Fig.  26.  Histogram  of  S3  Full  Baseline  Ar  Values 


8. 1 .3  Brownian  Motion  Drift  Analysis 

The  Brownian  motion  drift  method  requires  the  most  assumptions.  It  requires  the  difference  in  values  (between 
consecutive  observation  points)  to  be  independent  and  normally  distributed.  Thus,  if  we  are  to  use  the  r  values  for 
Brownian  motion,  then  the  Ar  values  must  be  independent  and  normally  distributed.  Unfortunately,  for  these 
examples  the  Ar  values  for  Si  and  S2  are  not  independent,  since  they  are  correlated,  as  demonstrated  in  Table  3. 
Therefore,  we  cannot  use  the  Brownian  motion  method  for  Si  and  S2.  The  Ar  values  for  S3  are  independent  and 
normally  distributed  as  described  in  Section  8.1.2,  so  we  can  use  the  Brownian  motion  drift  analysis  for  S3. 

8.2  Example  1 

The  examples  consider  that  a  regular  but  sparse  collection  of  data  is  ongoing.  Example  1  is  the  situation  where  all 
three  satellites  are  still  NOM.  Therefore,  there  are  no  cross-tags  present  as  seen  in  Fig.  27.  The  results  are  shown 
when  30  (i.e.  k  =  30)  new  observation  data  points  have  been  collected.  Note  that  the  use  of  k  =  30  is  only  meant  for 
illustration.  The  same  calculations  are  valid  for  any  value  of  k  >  0. 
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Fig.  27.  Example  1  Input  Data 


8.2. 1  Z-test  on  r  Values 

Fig.  28  -  Fig.  30  demonstrate  the  results  of  the  Z-test  for  dependent  data  on  r  values  for  Example  1.  Since  the  state  of 
all  three  satellites  remains  NOM,  we  expect  the  Z-test  for  dependent  data  to  determine  that  we  should  accept  the  null 
hypothesis,  H0.  By  looking  at  the  three  figures,  we  see  that  this  is  indeed  the  case  for  Satellite  1  and  Satellite  2. 
However,  Satellite  3  rejects  the  null  hypothesis  and  concludes  the  satellite  occurred  a  change  at  k=9  (or,  for  the 
cases  when  k  <  9,  this  test  would  have  determined  Satellite  3  to  be  NOM).  This  conclusion  can  be  further  explained 
by  referring  to  Fig.  15.  In  this  plot,  the  new  data’s  peaks  align  with  the  baseline’s  valleys,  so  the  method  is  unable  to 
detect  NOM  up  to  k  =  30. 
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Fig.  28.  Example  1  Si-i  Z-test  on  r  values  Results 
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Fig.  29.  Example  1  S2-2  Z-test  on  r  values  Results 
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Fig.  30.  Example  1  S3-3  Z-test  on  r  values  Results 


8.2.2  Classical  Sequential  Analysis 

Fig.  31  -  Fig.  33  demonstrate  the  results  of  the  classical  sequential  analysis  method  for  Example  1.  Since  the  state  of 
all  three  satellites  remains  NOM,  we  expect  the  classical  sequential  analysis  to  determine  we  should  accept  the  null 
hypothesis,  H0.  By  looking  at  the  three  figures,  we  see  that  this  is  the  case  for  all  the  satellites.  Thus,  the  classical 
sequential  analysis  provides  us  with  the  correct  results  for  Example  1 . 
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Fig.  31.  Example  1  Si-i  Classical  Sequential  Analysis  Results 


S2  Classical  Sequential  Analysis  Results 


Fig.  32.  Example  1  S2-2  Classical  Sequential  Analysis  Results 


Fig.  33.  Example  1  S3-3  Classical  Sequential  Analysis  Results 
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8.2.3  Brownian  Motion  Drift  Analysis 


Fig.  34  shows  the  results  of  the  Brownian  motion  drift  analysis  for  Example  1  for  S3.  Since  all  three  satellites  remain 
NOM,  we  expect  the  Brownian  motion  test  to  conclude  that  we  should  accept  H0  and  match  the  classical  sequential 
analysis  results.  From  Fig.  34,  we  see  this  is  the  case,  and  the  Brownian  motion  test  provides  us  with  consistent 
results  for  Example  1 . 


Fig.  34.  Example  1  S3-3  Brownian  Motion  Results 


8.2.4  Z-test  on  P(NOMIr)  Values 

Fig.  35  -  Fig.  37  demonstrate  the  results  of  the  Z-test  for  dependent  data  method  performed  on  the  P(NOMIr)  values 
for  Example  1.  Since  the  state  of  all  three  satellites  remains  NOM,  we  expect  the  Z-test  for  dependent  data  to 
determine  that  we  should  accept  the  null  hypothesis,  H0.  By  looking  at  the  three  figures,  we  see  that  this  is  the  case 
for  all  the  satellites.  Thus,  the  Z-test  for  dependent  data  on  the  P(NOMIr)  values  provides  us  with  the  correct  results 
for  Example  1 . 
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Fig.  36.  Example  1  S2-2  Z-test  on  P(NOMIr)  Results 


Fig.  37.  Example  1  S3-3  Z-test  on  P(NOMIr)  Results 


8.2.5  Data  Association 

Table  4  shows  the  results  for  the  three  tests  we  are  able  to  perform  on  all  three  satellites.  For  Si,  all  three  tests 
conclude  it  is  still  NOM,  so  we  will  accept  that  Si  is  NOM.  All  three  tests  also  conclude  S2  is  still  NOM,  so  we  will 
accept  that  it  is  also  NOM.  For  S3,  the  Z-test  for  dependent  data  on  r  values  concluded  it  was  ANOM.  Since  both  of 
the  other  two  satellites  are  NOM,  we  know  that  S3  is  not  cross-tagged  with  another  satellite  in  its  cluster,  but  we  do 
not  know  whether  the  satellite  is  truly  NOM  or  if  something  else  has  happened,  such  as  a  panel  offset  change,  to 
cause  the  satellite  to  now  be  ANOM.  This  is  addressed  in  Section  7. 


Table  4.  Example  1  Results  Chart 


Satellite 

r  Value  Z-test 

Classical  Sequential 
Analysis 

P(NOMIr)  Z-test 

Si 

NOM 

NOM 

NOM 

s2 

NOM 

NOM 

NOM 

S3 

ANOM 

NOM 

NOM 
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8.3  Example  2 


Example  2  is  where  the  baseline  data  for  Si  is  paired  with  the  new  data  from  S2  (orange  box),  which  we  will  denote 
as  S1-2  and  is  shown  in  Fig.  39.The  baseline  data  for  S2  is  paired  with  the  new  data  from  Si  (yellow  box),  denoted  as 
S2-1  as  shown  in  Fig.  40.  This  creates  a  cross-tag  between  Si  and  S2.  Fig.  38  depicts  the  new  input  data  for  Example 
2.  Fig.  39  and  Fig.  40  show  how  the  signature  of  the  baseline  satellite  matches  the  signature  of  the  satellite  of  the 
new  observations.  For  S1-2  and  S2-1,  the  signatures  do  not  have  significant  overlap,  so  we  expect  all  four  methods  to 
produce  accurate  results  because  the  r  and  P(  NOM  I  r  )  values  should  be  distinct.  Lastly,  the  baseline  of  S3  is  paired 
correctly  with  the  new  data  from  S3  (magenta  box),  so  S3  is  still  NOM. 
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Fig.  38.  Example  2  Input  Data 
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Fig.  39.  Si-2  Signatures 
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Fig.  40.  S2-1  Signatures 


8.3.1  Z-test  on  r  values 

The  Z-test  for  dependent  data  results  using  the  r  values  for  Example  2  are  shown  in  Fig.  41  -  Fig.  43.  Since  there  is  a 
cross-tag  between  Si  and  S2,  we  expect  to  accept  Ha  for  Si  and  S2  and  accept  H0  for  S3. The  Z-test  does  correctly 
conclude  Ha  for  Si  and  S2;  however,  as  in  Example  l,  S3  is  cross-tagged  with  itself,  so  the  reason  we  accept  Ha  is 
stated  in  Section  8.2.1. 
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Fig.  41.  Example  2  S1-2  Z-test  on  r  values  Results 
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Fig.  42.  Example  2  S2-1  Z-test  on  r  values  Results 
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Fig.  43.  Example  2  S3-3  Z-test  on  r  values  Results 


8.3.2  Classical  Sequential  Analysis 

The  classical  sequential  analysis  results  for  Example  2  are  shown  in  Fig.  44  -  Fig.  46.  Since  there  is  a  cross-tag 
between  Si  and  S2,  we  expect  to  accept  Ha  for  Si  and  S2  and  accept  H0  for  S3.  By  looking  at  the  three  figures,  we 
that  this  is  the  case.  Thus,  the  classical  sequential  analysis  provides  us  with  the  correct  results  for  Example  2. 
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Fig.  44.  Example  2  S1-2  Classical  Sequential  Analysis  Results 
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Fig.  45.  Example  2  S2-1  Classical  Sequential  Analysis  Results 


Fig.  46.  Example  2  S3-3  Classical  Sequential  Analysis  Results 
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8.3.3  Brownian  Motion  Drift  Analysis 


The  Brownian  motion  drift  analysis  result  for  Example  2  is  shown  in  Fig.  47.  S3  is  paired  with  itself,  so  we  expect  to 
accept  H0.  Fig.  47  indicates  this  is  the  case,  so  Brownian  motion  provides  the  correct  result  for  S3  by  concluding  that 
the  drift  is  zero. 


Fig.  47.  Example  2  S3-3  Brownian  Motion  Results 


8.3.4  Z-test  on  P(NOMIr)  Values 

The  Z-test  for  dependent  data  results  for  Example  2  are  shown  in  Fig.  48  -  Fig.  50.  Since  there  is  a  cross-tag  between 
Si  and  S2,  we  expect  to  accept  Ha  for  Si  and  S2  and  accept  H0  for  S3.  By  examining  the  three  figures  below,  we  see 
that  the  Z-test  for  dependent  data  performed  on  the  P(NOMIr)  values  accepts  Ha  for  Si  and  S2  and  accepts  H0  for  S3. 
Thus,  the  Z-test  for  dependent  data  provides  accurate  results  for  Example  2. 
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Fig.  49.  Example  2  S2-1  Z-test  on  P(NOMIr)  Results 


Fig.  50.  Example  2  S3-3  Z-test  on  P(NOMIr)  Results 


8.3.5  Data  Association 

Table  5  summarizes  the  results  for  the  three  main  tests.  For  Si  and  S2,  all  three  tests  conclude  ANOM,  so  we  accept 
that  they  are  both  ANOM.  For  S3,  two  tests  conclude  NOM  and  one  ANOM,  so  we  cannot  conclude  whether  S3  is 
NOM  or  ANOM.  To  try  to  resolve  the  change  in  satellites  Si  and  S2,  we  swap  their  new  data,  so  the  new  data  we 
were  using  for  Si  (orange  box)  we  will  use  for  S2  and  the  new  data  we  were  using  for  S2  (yellow  box)  we  will  use 
for  Si.  Thus,  we  result  with  the  same  situation  as  Example  1  as  seen  below  in  Fig.  51.  Therefore,  we  can  conclude 
that  in  the  original  data  for  Example  2  Si  and  S2  were  cross-tagged,  but  after  resolving  the  cross-tag,  Si  and  S2  are 
NOM  and  we  are  unsure  about  S3,  although  we  know  it  is  not  cross-tagged. 


Table  5.  Example  2  Results  Chart 


Satellite 

r  Value  Z-test 

Classical  Sequential 
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Fig.  51.  Example  2  Cross- tag  Resolution 


8.4  Example  3 


Example  3  contains  a  single  cross-tag  between  Si  and  S3,  so  the  baseline  data  for  Si  is  paired  with  the  new  data  from 
S3  (magenta  box),  S1-3  is  shown  in  Fig.  53,  and  the  baseline  data  for  S3  is  paired  with  the  new  data  from  Si  (yellow 
box);  S3-1  is  shown  in  Fig.  54.  S2,  in  this  case,  is  still  NOM.  The  input  data  for  example  3  is  illustrated  in  Fig.  52. 
Unlike  the  cross-tag  between  Si  and  S2, where  there  was  little  overlap  in  the  satellites’  signatures,  Fig.  53  and  Fig.  54 
show  there  is  some  overlap  for  the  signatures  of  Si  and  S3.  Therefore,  the  methods  in  this  section  might  have  a 
harder  time  distinguishing  between  the  two  satellites. 
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Fig.  52.  Example  3  Input  Data 
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Fig.  53.  Si-3  Signatures 


8.4. 1  Z-test  on  r  values 

The  Z-test  for  dependent  data  on  r  values  results  are  provided  in  Fig.  55  -  Fig.  57.  Because  Si  and  S3  are  cross- 
tagged,  we  anticipate  this  method  to  accept  Ha  for  Si  and  S3  and  accept  H0  for  S2.  Thus,  the  Z-test  for  dependent  data 
executed  on  r  values  provides  accurate  results  for  all  three  cases  in  Example  3. 
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Fig.  55.  Example  3  S1-3  Z-test  on  r  values  Results 
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Fig.  56.  Example  3  S2-2  Z-test  on  r  values  Results 
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Fig.  57.  Example  3  S3-1  Z-test  on  r  values  Results 


8.4.2  Classical  Sequential  Analysis 

Example  3  contains  a  cross-tag  between  Si  and  S3,  so  we  expect  to  accept  Ha  for  Si  and  S3  and  accept  H0  for  S2.  The 
classical  sequential  analysis  results  for  this  example  are  shown  in  Fig.  58  -  Fig.  60,  and  although  it  took  longer  to 
determine  S2  is  NOM,  we  get  the  results  we  expect. 
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Fig.  58.  Example  3  S1-3  Classical  Sequential  Analysis  Results 


Fig.  59.  Example  3  S2-2  Classical  Sequential  Analysis  Results 
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Fig.  60.  Example  3  S3-1  Classical  Sequential  Analysis  Results 
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8.4.3  Brownian  Motion  Drift  Analysis 


The  result  of  the  Brownian  drift  motion  analysis  for  Example  3  is  displayed  in  Fig.  61.  The  results  for  S3  accurately 
conclude  to  accept  Ha  and  confirm  that  there  is  a  non-zero  drift. 
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Fig.  61.  Example  3  S3-1  Brownian  Motion  Results 


8.4.4  Z-test  on  P(NOMIr)  Values 

The  results  for  the  Z-test  for  dependent  data  method  performed  on  the  P(NOMIr)  values  are  shown  in  Fig.  62  -  Fig. 
64.  In  this  example,  Si  and  S3  are  cross-tagged,  so  we  expect  the  test  to  accept  Ha  for  Si  and  S3.  Conversely,  S2 
receives  new  data  from  itself,  so  the  test  should  accept  the  null  hypothesis.  Displayed  below,  the  test  correctly 
concludes  to  accept  Ha  for  Si  and  accept  H0  for  S2.  The  graph  for  S3,  on  the  other  hand,  accepts  the  null  hypothesis, 
which  does  not  correspond  to  what  is  expected. 


Fig.  62  Example  3  S1-3  Z-test  on  P(NOMIr)  Results 
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Fig.  63  Example  3  S2-2  Z-test  on  P(NOMIr)  Results 
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Fig.  64  Example  3  S3-1  Z-test  on  P(NOMIr)  Results 


8.4.5  Data  Association 

Table  6  displays  the  overall  results  for  Example  3.  All  three  methods  determine  Si  is  ANOM  and  S2  is  NOM,  so  we 
will  accept  these  conclusions.  Only  two  methods  conclude  ANOM  for  S3,  so  we  are  unsure  of  the  state  of  S3.  To  try 
to  resolve  why  Si  is  ANOM  we  can  swap  its  new  data  (magenta  box)  with  the  new  data  of  S3  (yellow  box).  Since 
Example  3  was  created  by  swapping  the  new  data  of  Si  and  S3,  this  swap  will  correct  the  cross-tag,  and  we  will  have 
Example  1  again  as  seen  in  Fig.  65. 


Table  6.  Example  3  Results  Chart 
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Fig.  65.  Example  3  Cross-tag  Resolution 


8.5  Example  4 

Example  4  is  the  last  example  with  a  single  cross-tag.  The  cross-tag  is  between  S2  and  S3,  so  the  baseline  data  for  S2 
is  paired  with  the  new  data  from  S3  (magenta  box),  S2-3  shown  in  Fig.  67,  and  the  baseline  data  for  S3  is  paired  with 
the  new  data  from  S2  (orange  box),  S3-2  shown  in  Fig.  68.  Thus,  Si  is  NOM.  The  cross-tag  between  S2  and  S3  has 
considerable  overlap  for  the  signatures  of  S2  and  S3  as  can  be  seen  in  Fig.  67  and  Fig.  68.  Therefore,  the  methods 
have  a  harder  time  distinguishing  between  the  two  satellites,  and  consequently,  they  might  not  be  able  to  detect  the 
cross-tag. 
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Fig.  66.  Example  4  Input  Data 
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Fig.  67.  S2-3  Signatures 


Fig.  68.  S3-2  Signatures 


8.5.1  Z-test  on  r  values 

The  results  for  the  Z-test  for  dependent  data  on  r  values  are  shown  in  Fig.  69  -  Fig.  71.  Because  S2  and  S3  are  cross- 
tagged  in  Example  4,  our  test  is  correct  in  concluding  to  accept  Ha  for  S2  and  S3  and  to  accept  H0  for  Si.  S2-3  does 
take  more  than  the  expected  number  of  observations  to  reach  a  decision,  but  the  reasoning  for  this  is  that  the 
signatures  are  very  similar  to  one  another  as  seen  in  Fig.  67. 
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Fig.  69.  Example  4  Sm  Z-test  on  r  values  Results 
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Fig.  70.  Example  4  S2-3  Z-test  on  r  values  Results 
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Fig.  71.  Example  4  S3-2  Z-test  on  r  values  Results 


8.5.2  Classical  Sequential  Analysis 

For  Example  4,  we  expect  to  accept  Ha  for  S2  and  S3  and  to  accept  H0  for  Si,  because  S2  and  S3  are  cross-tagged. 
Unfortunately,  Fig.  72  -  Fig.  74  show  that  the  classical  sequential  analysis  could  not  find  the  cross-tag  for  S2-3  or  S3- 
2.  However,  for  S3-2,  if  we  update  the  slider  by  moving  it  forward  to  halfway  between  the  start  of  the  new  data  and 
the  initial  conclusion  to  accept  H0  and  then  restart  the  method  over  again,  the  second  run  of  the  classical  sequential 
analysis  accepts  Ha  at  observation  10.  Similarly,  if  we  update  the  time  slider  for  S2-3,  we  would  eventually  obtain  the 
conclusion  to  accept  Ha  at  observation  24,  but  it  takes  several  updates  of  the  time  slider.  The  classical  sequential 
analysis  has  difficulty,  especially  for  S2-3,  because  of  the  orbit  angle  of  the  new  observations.  Observations  0  through 
22  are  in  the  orbit  angle  range  -75  to  -45,  and  Fig.  67  illustrates  that  the  signatures  of  S2  and  S3  at  these  orbit  angles 
are  very  similar.  This  causes  the  r  values  that  we  are  testing  to  be  similar,  so  the  test  is  unable  to  detect  a  change. 


Fig.  72.  Example  4  S1-1  Classical  Sequential  Analysis  Results 
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S2  Classical  Sequential  Analysis  Results 
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Fig.  73.  Example  4  S2-3  Classical  Sequential  Analysis  Results 
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Fig.  74.  Example  4  S3-2  Classical  Sequential  Analysis  Results 


8.5.3  Brownian  Motion  Drift  Analysis 


The  result  for  Example  4  for  the  Brownian  motion  drift  analysis  is  better  for  S3  than  the  classical  sequential  analysis 
results.  Unlike  the  classical  sequential  analysis  method,  where  we  had  to  update  the  time  slider,  we  can  accept  Ha  for 
S3  the  first  time  we  run  the  test. 
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S3  Brownian  Motion  Results 
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Fig.  75.  Example  4  S3-2  Brownian  Motion  Results 


8.5.4  Z-test  on  P(NOMIr)  Values 

In  Example  4,  S2  and  S3  are  cross-tagged,  so  the  Z-test  for  dependent  data  on  P(NOMIr)  values  results  are  expected 
to  accept  H0  for  Si  and  accept  Ha  for  S2  and  S3.  The  results  for  this  method  are  shown  in  Fig.  76  -  Fig.  78.  The  test 
for  this  example  correctly  concludes  H0  for  Si  and  Ha  for  S2  and  S3.  Although  it  does  take  S3  more  observations  than 
expected  to  come  to  a  decision,  we  are  attributing  this  to  the  similarities  in  their  signatures  in  Fig.  68. 
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Fig.  76.  Example  4  S1-1  Z-test  on  P(NOMIr)  Results 
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Fig.  77.  Example  4  S2-3  Z-test  on  P(NOMIr)  Results 


8.5.5  Data  Association 

Table  7  displays  the  results  for  Example  4.  The  only  conclusion  all  three  tests  agree  on  is  that  Si  is  NOM.  For  both 
S2  and  S3  the  classical  sequential  analysis  method  determines  NOM,  while  the  other  two  methods  determine  ANOM. 
Thus,  the  only  conclusion  we  can  come  to  is  that  Si  is  NOM.  If  we  want  to  try  to  resolve  the  potential  change  of  S2 
and  S3,  then  we  could  try  and  swap  their  new  data  and  run  the  tests  again.  Upon  swapping  the  new  data,  we  actually 
would  reverse  the  cross-tag  created  for  Example  4,  and  therefore,  we  would  again  end  up  with  Example  1.  The  data 
swap  is  illustrated  in  Fig.  79. 


Table  7.  Example  4  Results  Chart 
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Fig.  79.  Example  4  Cross-tag  Resolution 


8.6  Example  5 

Example  5  contains  a  multi-satellite  cross-tag.  The  baseline  data  for  Si  is  paired  with  the  new  data  from  S2  (orange 
box),  denoted  as  S1-2,  the  baseline  data  for  S2  is  paired  with  the  new  data  from  S3  (magenta  box),  denoted  as  S2-3,  and 
the  baseline  data  for  S3  is  paired  with  the  new  data  from  Si  (yellow  box),  denoted  as  S3-1.  Therefore,  there  are  no 
NOM  satellites  in  this  example. 
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Fig.  80.  Example  5  Input  Data 


8.6. 1  Z-test  on  r  values 

For  the  Z-test  for  dependent  data  on  r  values,  the  results  are  displayed  in  Fig.  81  -  Fig.  83.  This  method  has  three 
different  cross-tagged  situations,  and  the  test  accurately  accepts  the  alternative  hypothesis  for  Si,  S2,  and  S3. 
Therefore,  the  Z-test  for  dependent  data  on  r  values  is  correct  for  all  cases  in  Example  5. 
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Fig.  81.  Example  5  S1-2  Z-test  on  r  values  Results 


S2  Z-test  (r  value)  Results 
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Fig.  82.  Example  5  S2-3  Z-test  on  r  values  Results 


S3  Z-test  (r  value)  Results 
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Fig.  83.  Example  5  S3-1  Z-test  on  r  values  Results 
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8.6.2  Classical  Sequential  Analysis 


The  classical  sequential  analysis  results  for  Example  5  are  shown  in  Fig.  84  -  Fig.  86.  Although  we  expect  to  accept 
Ha  for  all  three  satellites,  we  only  accept  Ha  for  Si  and  S3.  The  classical  sequential  analysis  method  is  unable  to 
determine  the  cross-tag  for  S2.  The  reasons  for  this  are  discussed  in  Section  8.5.2. 
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Fig.  84.  Example  5  S1-2  Classical  Sequential  Analysis  Results 
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Fig.  85.  Example  5  S2-3  Classical  Sequential  Analysis  Results 
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S3  Classsical  Sequential  Analysis  Results 
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Fig.  86.  Example  5  S3-1  Classical  Sequential  Analysis  Results 


8.6.3  Brownian  Motion  Drift  Analysis 

The  result  for  Example  5  for  Brownian  motion  drift  analysis  is  seen  in  Fig.  87.  The  Brownian  motion  drift  analysis 
correctly  accepts  Ha  for  S3. 
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Fig.  87.  Example  5  S3-1  Brownian  Motion  Results 


8.6.4  Z-test  on  P(NOMIr)  Values 

The  results  for  the  Z-test  for  dependent  data  on  P(NOMIr)  values  are  shown  in  Fig.  88  -  Fig.  90.  The  test  correctly 
identifies  ANOM  for  Si  and  S2.  However,  the  Z-test  for  dependent  data  accepts  H0  for  S3,  which  is  not  consistent 
with  the  results  the  test  would  expect. 


AMOS  Technical  Conference  2015 


52 


Detects  ANOM  at 
Observation  #1 


— z  sscore 
-«-P(NOM|r)  values 


Fig.  89.  Example  5  S2-3  Z-test  on  P(NOMIr)  Results 


Fig.  90.  Example  5  S3-1  Z-test  on  P(NOMIr)  Results 
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8.6.5  Data  Association 


Table  8  summarizes  the  results  for  Example  5.  All  tests  conclude  Si  is  ANOM,  but  they  do  not  all  agree  for  S2  and 
S3.  Therefore,  we  can  only  conclude  Si  is  ANOM.  To  try  and  resolve  that  Si  is  ANOM,  we  can  swap  its  new  data 
(orange  box)  with  one  of  the  other  two  satellites.  Unfortunately,  both  of  the  other  satellites  have  two  ANOM  results 
and  one  NOM  result,  so  there  is  no  clue  as  which  one  to  swap  with  Si.  Let  us  swap  the  new  data  of  Si  (orange  box) 
with  the  new  data  of  S2  (magenta  box).  Since  Si  is  cross-tagged  with  S2,  upon  swapping  the  data  and  running  the 
tests  again,  we  would  now  conclude  S2  is  NOM.  Since  the  new  data  of  S2  was  S3,  when  we  swap  the  data  Si 
becomes  cross-tagged  with  S3  instead  of  S2.  S3  is  already  cross-tagged  with  Si,  so  we  have  Example  3  again.  Section 
8.4.5  describes  how  to  resolve  Example  3.  Once  the  Example  3  cross-tag  is  resolved,  Fig.  91  demonstrates  the  case 
of  Example  1. 


Table  8.  Example  5  Results  Chart 
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Fig.  91.  Example  5  Cross- tag  Resolution 


8.7  Example  6 


Example  6  contains  a  multi-satellite  cross-tag.  The  baseline  data  for  Si  is  paired  with  the  new  data  from  S3  (magenta 
box),  denoted  as  S1-3,  the  baseline  data  for  S2  is  paired  with  the  new  data  from  Si  (yellow  box),  denoted  as  S2-1,  and 
the  baseline  data  for  S3  is  paired  with  the  new  data  from  S2  (orange  box),  denoted  as  S3-2.  Thus,  there  are  no  NOM 
satellites  in  this  example.  Fig.  92  illustrates  the  new  input  data  for  Example  6. 
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Fig.  92.  Example  6  Input  Data 


8.7. 1  Z-test  on  r  values 

Example  6  results  for  the  Z-test  for  dependent  data  on  r  values  are  shown  in  Fig.  93  -  Fig.  95.  All  three  cases  are 
expected  to  accept  the  alternative  hypothesis.  As  seen  below,  Si,  S2  and  S3  conclude  to  accept  Ha.  Thus,  the  Z-test 
for  dependent  data  on  r  values  is  accurate  for  Example  6. 


Fig.  93.  Example  6  S1-3  Z-test  on  r  values  Results 
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Fig.  95.  Example  6  S3-2  Z-test  on  r  values  Results 


8.7.2  Classical  Sequential  Analysis 

The  classical  sequential  analysis  results  for  Example  6  are  displayed  in  Fig.  96  -  Fig.  98.  Although  we  expect  all 
three  cases  to  conclude  ANOM,  the  test  for  S3  accepts  H0.  The  reasons  for  this  are  described  in  Section  8.5.2. 
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Fig.  96.  Example  6  S1-3  Classical  Sequential  Analysis  Results 
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Fig.  97.  Example  6  S2-1  Classical  Sequential  Analysis  Results 


S3  Classical  Sequential  Analysis  Results 
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Fig.  98.  Example  6  S3-2  Classical  Sequential  Analysis  Results 


8.7.3  Brownian  Motion  Drift  Analysis 

The  Brownian  motion  drift  analysis  result  for  Example  6  is  shown  below  in  Fig.  99.  Brownian  motion  correctly 
accepts  Ha,  so  its  results  for  S3  are  better  than  those  of  the  classical  sequential  analysis. 
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8.7.4  Z-test  on  P(NOMIr)  Values 

In  this  example,  the  Z-test  for  dependent  data  on  P(NOMIr)  values  accepts  the  alternative  hypothesis  for  Si,  S2  and 
S3.  Because  all  three  satellites  have  been  cross-tagged  with  one  another,  this  method  provides  accurate  results  for  all 
three  cases.  The  S3-2  cross-tag  situation  does  take  significantly  longer  to  detect  ANOM  compared  to  the  other  two 
cases.  However,  this  is  a  result  of  the  similar  signatures  graphs  in  Fig.  68.  The  results  for  the  Z-test  for  dependent 
data  on  P(NOMIr)  values  are  shown  in  Fig.  100  -  Fig.  102. 


Fig.  100.  Example  6  S1-3  Z-test  on  P(NOMIr)  Results 
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Fig.  101.  Example  6  S2-1  Z-test  on  P(NOMIr)  Results 


Fig.  102.  Example  6  S3-2  Z-test  on  P(NOMIr)  Results 


8.7.5  Data  Association 

The  overall  results  for  Example  6  are  displayed  in  Table  9.  All  three  tests  find  Si  and  S2  to  be  ANOM,  so  we  can 
conclude  Si  and  S2  are  ANOM.  Unfortunately,  the  classical  sequential  analysis  method  finds  S3  to  be  NOM,  while 
the  other  two  methods  determine  it  to  be  ANOM.  Therefore,  we  cannot  make  any  conclusions  about  S3.  If  we  want 
to  try  to  determine  the  cause  for  change  in  Si  and  S2,  then  we  can  swap  their  new  data.  Since  S2  is  cross-tagged  with 
Si  (yellow  box),  its  new  data  is  truly  the  new  data  from  Si.  So  when  we  swap  their  new  data  and  run  the  tests  again, 
Si  becomes  NOM.  Then  the  new  data  of  Si  is  S3  (magenta  box),  so  after  swapping  the  data,  S2  becomes  cross-tagged 
with  S3.  Thus,  this  becomes  equivalent  to  Example  4.  The  description  of  how  to  resolve  Example  4  is  in  Section 
8.5.5.  A  depiction  of  the  cross-tag  resolution  for  Example  6  is  shown  in  Fig.  103. 


Table  9.  Example  6  Results  Chart 
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Fig.  103.  Example  6  Cross-tag  Resolution 


8.8  Summary  of  Cross-tag  resolution 

The  six  examples  illustrate  how  the  New  Data  for  a  cluster  of  three  satellites  may  present  itself  with  different 
combinations  of  cross-tags  between  satellites.  The  cross-tag  resolution  procedure  comprised  a  successive  and 
adaptive  application  of  the  Z-test  and  sequential  analysis  based  on  the  preceding  results  of  the  composite  hypothesis 
testing.  The  resolution  of  the  situations  in  examples  2  through  4  required  a  two  stage  application  of  the  composite 
hypothesis  testing,  while  examples  5  and  6  required  three  stages.  In  each  case,  the  cross -tag  situations  were  reduced 
to  the  data  analysis  performed  in  Example  1 .  Therefore,  although  our  original  conclusions  for  an  example  might  be 
that  multiple  satellites  are  ANOM,  after  resolving  the  cross-tags,  we  will  conclude  that  Si  and  S2  are  NOM,  and  we 
cannot  make  a  conclusion  about  S3.  Thus,  the  three  methods  used  in  the  examples  and  results  tables  provided  to  be 
useful  in  all  cases.  The  assessment  is  based  on  the  use  of  P(type  I  error)  =  0.05  and  P(type  II  error)  =  0.05.  For  the 
composite  hypothesis  test,  the  resultant  P(type  I  error)  =  3*0.05  or  15%. 

Thus,  there  are  several  conditions  when  the  results  of  the  three  tests  do  not  agree  with  each  other.  The  assessment  of 
NOM  or  ANOM  is  performed  only  when  all  tests  agree.  To  improve  the  test  results  (this  is  so  that  all  tests  agree 
with  each  other),  we  would  first  need  to  improve  the  Inversion  Model  and  Prediction  Model  to  obtain  more  accurate 
r  and  P(NOMIr)  values  to  provide  to  the  tests.  To  adjust  the  two  models,  we  need  more  observations  for  each 
satellite,  especially  S3.  We  also  need  new  observations  in  order  to  detect  the  occurrence  of  ANOM  on  an  ongoing 
basis.  In  the  present  work,  the  Inversion  Model  and  Predictive  Model  use  the  same  physics  procedure  based  on  the 
two  facet  model  [1].  Thus,  the  improvement  of  the  Inversion  Model  also  results  in  the  improvement  of  the  Predictive 
Model.  This  is  described  in  Section  9.0. 

8.9  Update  of  Baseline  Signature 

Once  a  satellite  is  determined  to  be  NOM  by  the  composite  hypothesis  test,  we  can  update  the  baseline  for  that 
particular  satellite.  Updating  the  baseline  is  essential  to  be  able  to  account  for  seasonal  changes.  After  the  baseline 
has  been  updated,  testing  is  restarted  to  keep  a  current  status  of  the  satellite.  Fig.  104  shows  a  notional  example  of 
updating  the  baseline  for  Si.  Since  S3  was  determined  to  be  ANOM  in  Example  1,  we  cannot  truly  update  the 
baseline  for  the  satellites  in  the  cluster.  However,  this  section  demonstrates  how  the  baseline  could  be  updated  if  the 
conditions  were  satisfied. 

The  dotted  lines  in  Fig.  104  show  the  movement  of  the  time  slider.  Because  the  classical  sequential  analysis  method 
determined  Si  was  NOM  at  observation  7  in  Example  1,  the  time  slider  is  forwarded  7  points.  It  is  up  to  the  user  to 
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decide  how  far  forward  the  time  slider  is  moved.  The  slider  can  be  moved  forward  to  any  point  in  the  region  from 
the  start  of  the  new  data  up  to  where  the  classical  sequential  analysis  method  determines  the  satellite  is  NOM. 


Sx  Apparent  Magnitude  Over  Time 


Relative  Time  (hours) 


♦Baseline 
■New  Data 


Fig.  104.  Si  Update  of  Baseline  Signature 


After  move  the  time  slider  forward  7  observations,  we  can  then  use  any  of  the  methods  described  in  Section  4.0  to 
begin  reevaluating  the  new  data.  Fig.  105  shows  the  results  of  the  classical  sequential  analysis  method  ran  on  Si 
using  the  updated  baseline.  The  classical  sequential  analysis  again  determines  Si  is  NOM.  Thus,  we  could  move  the 
time  slider  forward  once  again  and  restart  the  process. 


Updated  Sx  Classical  Sequential  Analysis 
Results 


Fig.  105.  Si  Updated  Classical  Sequential  Analysis  Results 


9.0  Sensor  Tasking  for  Fine  Feature  Characterization 

For  the  examples,  our  decision  criterion  did  not  lead  the  user  to  a  definitive  conclusion  that  S3  was  NOM  or  ANOM. 
This  work  considers  that  such  a  lack  of  conclusion  can  be  due  to  two  reasons.  First,  the  fidelity  of  the  Predictive 
Model  may  be  too  low.  This  results  in  the  values  of  Brightness  Ratio  to  exhibit  significant  scatter.  Second,  the 
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satellite  may  have  changed,  which  results  in  a  lower  fidelity  for  the  Predictive  Model  since  it  is  derived  from  the 
inversion  of  baseline  data.  Or,  the  Predictive  Model  may  have  low  fidelity  and  the  satellite  has  changed. 
Accordingly,  the  goal  of  sensor  tasking  is  to  collect  data  so  as  to  enable  the  dual  goals  of  assessment  of  change 
and/or  improvement  of  Predictive  Model  as  efficiently  as  possible. 

The  motivation  for  the  tasking  method  is  from  the  maximum  information  entropy  principle,  which  expresses  the 
claim  of  maximum  ignorance.  The  probability  distribution  function  for  a  quantity  of  interest  is  chosen  to  be  one  that 
makes  least  claim  to  being  informed  beyond  the  baseline  data  [4].  The  quantity  of  interest  in  sensor  tasking  is  the 
definition  of  observation  conditions;  namely  the  direction  of  illumination  and  the  view  direction  for  the  sensor.  In 
this  work,  it  is  assumed  that  the  effect  of  the  illumination  and  observation  conditions  is  captured  by  the  value  of  a 
single  entity,  namely  the  orbit  angle.  The  orbit  angle  is  defined  to  be  the  projection  of  the  sun-satellite-sensor  angle 
in  the  orbital  plane  of  the  satellite.  It  is  similar  to  the  longitudinal  phase  angle,  only  difference  being  that  the 
longitudinal  phase  angle  is  computed  with  respect  to  the  equatorial  plane.  Or,  for  a  satellite  with  a  zero  orbital 
inclination,  the  orbit  angle  is  same  as  the  longitudinal  phase  angle.  This  is  a  coarse  yet  practical  approximation 
because  a  complete  definition  of  the  illumination  and  observation  directions  is  a  function  of  the  geometry  and 
orientation  of  the  satellite  solar  panel  and  its  body.  At  a  minimum,  it  consists  of  four  angles  [5]. 

We  consider  that  the  change  may  occur  such  that  it  may  not  be  observable  from  any  value  of  the  orbit  angle.  This  is 
because  the  change  may  cause  minimal  net  change  in  the  projected  geometry  of  the  satellite  with  respect  to  the  sun 
and  the  sensor.  The  orbit  angle  at  which  the  change  may  be  manifested  in  terms  of  difference  in  its  brightness  is 
function  of  the  change  itself  and  is  unknown  a  priori.  Thus,  the  sensor  tasking  for  the  purpose  of  change  detection 
can  make  no  assumption  with  respect  to  the  choice  of  the  orbit  angle.  Alternately,  the  choice  of  an  orbit  angle  for  the 
next  observation  needs  to  be  chosen  as  per  a  uniform  probability  distribution  function.  This  strategy  is  postulated  to 
maximize  the  amount  of  new  information  per  observation  as  per  the  principle  of  maximum  information  entropy. 

We  also  note  that  the  fidelity  of  the  Predictive  Model  varies  with  the  orbit  angle.  This  is  manifested  in  the  random 
character  of  the  Brightness  Ratio  versus  orbit  angle  as  shown  in  Fig.  2  -  Fig.  4.  The  range  of  values  for  the 
Brightness  Ratio  is  larger  for  S3  as  compared  to  Si  and  S2.  This  is  related  to  the  reflection  phenomenology  as  a 
function  of  the  orbit  angle.  For  example,  at  intermediate  values  of  orbit  angle,  the  satellite  brightness  is  commonly 
governed  by  diffuse  reflection.  This  is  because  the  specular  behavior  of  the  solar  panels  is  at  smaller  orbit  angles 
(closer  to  0°),  and  the  specular  behavior  off  the  fine  body  features  is  typically  at  larger  orbit  angles  (closer  to  90°).  It 
is  generally  easier  to  attain  higher  fidelity  for  the  Predictive  Model  under  diffuse  reflection  conditions  at 
intermediate  orbit  angles  rather  than  its  values  closer  to  0°  or  90°.  This  is  particularly  when  there  is  no  self-occlusion 
caused  by  any  fine  features  on  the  body  in  the  projected  view  of  the  satellite  with  respect  to  the  sensor. 

We  also  note  that  the  fidelity  of  the  Predictive  Model  varies  with  the  orbit  angle.  This  is  manifested  in  the  random 
character  of  the  Brightness  Ratio  versus  orbit  angle  as  shown  in  Fig.  106  -  Fig.  108.  The  range  of  values  for  the 
Brightness  Ratio  is  larger  for  S3  as  compared  to  Si  and  S2.  This  is  related  to  the  reflection  phenomenology  as  a 
function  of  the  orbit  angle.  For  example,  at  intermediate  values  of  orbit  angle,  the  satellite  brightness  is  commonly 
governed  by  diffuse  reflection.  This  is  because  the  specular  behavior  of  the  solar  panels  is  at  smaller  orbit  angles 
(closer  to  0°),  and  the  specular  behavior  off  the  fine  body  features  is  typically  at  larger  orbit  angles  (closer  to  90°).  It 
is  generally  easier  to  attain  higher  fidelity  for  the  Predictive  Model  under  diffuse  reflection  conditions  at 
intermediate  orbit  angles  rather  than  its  values  closer  to  0°  or  90°.  This  is  particularly  when  there  is  no  self-occlusion 
caused  by  any  fine  features  on  the  body  in  the  projected  view  of  the  satellite  with  respect  to  the  sensor. 

Consider  the  situation  when  there  was  no  change  in  the  satellite.  In  such  a  case,  the  lack  of  conclusion  in  regards  to 
NOM  or  ANOM  for  a  satellite  would  depend  on  the  fidelity  of  the  Predictive  Model.  This  is  because  the  Predictive 
Model  is  utilized  to  predict  the  expected  brightness  of  the  satellite  at  point  in  the  new  data.  If  the  fidelity  of  the 
Predictive  Model  is  insufficient,  the  computed  values  of  the  Brightness  Ratio  at  the  new  data  points  can  be  such  that 
test  statistic  for  the  new  data  suggests  a  difference  from  the  baseline  data.  This  may  be  the  case  even  though  there  is 
no  statistically  significant  change. 

The  Predictive  Model  could  be  improved  when  additional  data  is  collected.  Any  collection  of  additional  data  needs 
to  be  such  that  it  maximizes  new  information.  In  this  regard,  the  fidelity  of  the  Predictive  Model  as  a  function  of  the 
orbit  angle  is  postulated  as  being  inversely  related  to  the  new  information  generated  if  the  sensor  tasking  was 
defined  to  collect  new  data  at  that  orbit  angle.  This  postulation  may  be  interpreted  by  considering  two  cases  for  the 
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orbit  angle,  one  where  the  fidelity  of  Predictive  Model  is  high  and  the  second  where  it  is  low.  This  may  be  idealized 
further  as  intervals  of  orbit  angle  values  where  the  Brightness  Ratio  equals  zero  and  nonzero,  respectively.  When  the 
Brightness  Ratio  is  zero,  the  expected  value  of  satellite  brightness  is  equal  to  the  predicted  value,  or  there  is  no  new 
information  in  regards  to  the  improvement  of  the  Predictive  Model.  When  the  Brightness  Ratio  is  nonzero,  there  is 
new  information  that  is  useful  in  order  to  improve  the  Predictive  Model.  Thus,  it  is  postulated  that  the  sampling  of 
orbit  angle  values  during  the  collection  of  additional  data  be  inversely  proportional  to  the  absolute  value  of  the 
Brightness  Ratio  in  order  to  maximize  new  information  per  observation. 


S2  r  Values  Over  Orbit  Angle 
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Fig.  107.  Satellite  S2  r  Values  Versus  Orbit  Angle 
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S3  r  Values  Over  Orbit  Angle 
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Fig.  108.  Satellite  S3  r  Values  Versus  Orbit  Angle 


Since  S3  is  the  satellite  for  which  we  require  more  data,  we  will  focus  on  S3  for  the  rest  of  this  section.  If  the  sensor 
was  utilized  in  a  staring  mode  (i.e.  persistent  observation),  the  sampling  rate  for  the  observation  data  would  be 
uniform  along  the  orbit  angle  axis.  In  such  a  case,  the  total  new  information  would  be  proportional  to  the  area  under 
the  curve  of  orbit  angle  versus  the  absolute  value  of  the  Brightness  Ratio  (i.e.,  r  values).  Accordingly,  Fig.  109 
shows  a  plot  of  new  information  versus  orbit  angle  for  S3.  The  peaks  of  this  graph  are  areas  where  the  Predictive 
Model  has  lower  fidelity,  while  the  valleys  are  areas  where  the  model  has  higher  fidelity.  Fig.  110  illustrates  how  the 
new  information  would  accumulate  if  the  sensor  was  to  observe  the  satellite  persistently.  This  graph  is  normalized  to 
a  maximum  value  of  1.0,  which  is  meant  to  represent  the  totality  of  new  information.  We  broke  the  total  new 
information  into  10%  bins,  which  is  meant  to  represent  that  persistent  observations  over  each  orbital  angle  bin 
would  have  the  ability  to  provide  10%  of  the  total  new  information.  These  bins  are  displayed  in  Fig.  111.  Comparing 
Fig.  109  and  Fig.  Ill,  the  smaller  bins  align  with  the  areas  with  the  most  error. 
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Fig.  109.  New  information  as  a  function  of  orbit  angle 
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Fig.  110.  Accumulation  of  new  information  as  a  function  of  orbit  angle 
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Fig.  111.  Orbit  Angle  Bins  for  New  Data  Collection 


Consider  a  situation  when  the  Predictive  Model  has  limited  fidelity  and  the  satellite  may  have  changed.  For 
example,  the  lower  fidelity  of  the  Predictive  Model  could  be  caused  by  self-occlusion  due  to  the  fine  features.  Then 
the  sensor  tasking  would  have  a  joint  goal  of  maximizing  new  information  per  observation  for  the  purpose  of 
improving  the  Predictive  Model  and/or  detect  change.  The  corresponding  tasking  strategy  can  be  defined  to  combine 
the  needs  of  the  Predictive  Model  and  change  detection  using  a  two-step  approach  as  follows  (Fig.  Ill): 

•  The  probability  of  selection  of  an  orbital  bin  is  equal.  This  is  to  maximize  new  information  with  respect  to 
change  detection. 

•  The  probability  of  selection  of  an  observation  condition  within  an  orbital  bin  is  uniformly  distributed.  This 
is  to  maximize  new  information  with  respect  to  the  improvement  of  fidelity  of  the  Predictive  Model. 

The  characterization  of  fine  features  using  such  a  two-step  sensor  tasking  strategy  is  a  part  of  our  future  work. 


10.0  Ongoing  work  and  closure 

This  paper  provides  a  statistical  assessment  technique  for  the  resolution  of  multi -satellite  cross-tag  by  making  an 
adaptive  use  of  sequential  analysis  techniques.  The  method  for  cross-tag  detection  and  resolution  is  useful 
irrespective  of  the  different  types  of  cross-tag  scenarios  feasible  for  a  cluster  and  when  the  signatures  of  the  satellites 
have  significant  overlap.  Although  the  technique  is  demonstrated  for  a  three  satellite  cluster,  it  can  be  extended  to 
clusters  containing  a  larger  number  of  satellites. 

The  current  work  uses  simplified  composite  hypothesis  testing,  which  allows  a  decision  to  be  made  only  when  all 
tests  agree.  The  next  step  is  to  enhance  the  composite  test  method  by  calculating  the  probabilities  of  all  possible 
incorrect  conclusions,  so  that  the  user  can  make  more  informed  decisions  about  the  status  of  the  satellites.  This  will 
likely  involve  simulation  due  to  the  dependency  of  the  hypothesis  tests  involved.  . 

The  current  analyses  assume  an  idealized  cross-tag  situation  where  the  cross-tag  in  the  new  data  does  not  change 
unless  it  is  rectified.  The  next  step  is  to  extend  the  statistical  assessment  to  the  case  of  a  switching  or  time-dependent 
cross-tag.  This  is  when  a  single  set  of  new  data  for  a  satellite  is  cross-tagged  with  different  satellites. 

The  current  Inversion  Model  and  Prediction  Model  use  a  0th  order  two-facet  model.  The  next  step  is  to  enhance  the 
models  to  include  the  contribution  of  new  data  collected  as  per  the  principle  of  maximum  information  entropy  in 
order  to  better  account  for  the  antennae,  dishes  and  self-occlusions. 
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13.0  Appendix  A:  Notation 


Text  notation: 


Brightness:  Value  of  Brightness  Data  at  a  single  data  point 

Brightness  Data:  Single  point  brightness  data  point  collected  by  a  ground  or  space-based  sensor  during  the  routine 
synoptic  search  operation.  This  data  is  collected  along  with  the  angles  only  metric  data 
Brightness  Ratio:  Ratio  of  observed  Brightness  to  predicted  Brightness 
Body:  Body  of  a  GEO  satellite 

Change:  Modification  of  state  of  a  satellite  from  NOM  to  ANOM  or  vice  versa. 

Change  Detection:  To  recognize  that  the  state  of  satellite  has  undergone  Change 
Cluster  Peer:  A  pair  of  satellite  in  a  cluster  can  be  cross-tagged 

NOM:  Nominal  status  of  a  satellite.  This  is  when  the  correlation  coefficient  between  the  observed  Brightness  and 
the  predicted  Brightness  exceeds  a  user-defined  threshold  limit. 

ANOM:  Anomalous  status  of  a  satellite.  This  is  when  the  correlation  coefficient  between  the  observed  Brightness 
and  the  predicted  Brightness  is  below  the  user-defined  threshold  limit  for  NOM. 

Panel:  Panel  term  for  a  GEO  satellite  (it  combines  the  effect  of  both  solar  panels  into  a  single  term) 

PDF:  Probability  distribution  function 
CDF:  Cumulative  distribution  function 

Signature  Data:  A  sequence  of  Brightness  measurements  collected  by  a  dedicated  sensor  during  a  single  pass  for  a 
target  satellite.  For  GEO  satellites,  such  data  is  collected  at  a  frame  rate  such  as  one  data  point  per  minute,  etc. 

Mathematics  notation: 

rk  =  ratio  of  observed  Brightness  of  the  satellite  at  pass  k  to  predicted  Brightness  of  the  satellite  at  pass  k.  This  ratio 
is  defined  only  at  the  orbital  location  when  Brightness  data  is  collected 
Arfcis  the  difference  between  rk  and  rk_x 

f  denotes  the  adjusted  r  value  after  it  has  subtracted  p  and  divided  by  a 

k  =  Index  for  an  orbital  pass  number.  The  time  slider  origin  is  k  =  0.  Prior  data  is  for  k  <  0.  New  data  is  for  k  >  0. 

Iok  is  the  observed  single  point  brightness 

IMk  is  the  expected  single  point  brightness 

E(rk )  denotes  the  expected  value 

a  =  the  level  of  significance  of  the  test  =  P(Type  I  Error) 

/?  =  P(Type  II  Error) 

8  is  the  number  of  standard  deviations  that  mean  drift  mu  is  away  from  null  hypothesis  mean  drift  0 

P(NOMIrk)  =  Probability  that  satellite  is  NOM  after  pass  k  given  the  Brightness  Ratio  rk 

H0  denotes  the  null  hypothesis 

Ha  denotes  the  alternative  hypothesis 

p  is  the  mean  for  the  prior  data 

o  is  the  standard  deviation  for  the  prior  data 

Ln  denotes  the  use  of  a  likelihood  function 
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